Repository: GoogleCloudPlatform/spark-on-k8s-operator
Pull request #914 (2020-05-15 13:07:45)

Support changing port in ui exposition
This PR makes it possible to change and customize:

- Service object : By allowing to specify a `port` different from `targetport`
- Ingress object : By allow adding annotations + adding tls specification (optional)

By default only the port `4040` or a port specified in `spark.ui.port `is exposed with a basic configuration of the Service and Ingress objects. 

It turns out, in several companies spark applications run in some private network which is usually accessible with limited number of ports exposed + using some ingress controllers to redirect trafic within the private network. For instance, we may have only ports `443` and `80` exposed to the ingress controller. These ports could never be defined in the spark configuration since they are below than 1024. Though the spark app fails at the start time. A common solution, is to expose a specific port at the ingress object which then redirects to the Service. The service listens on the same port used by the ingress object. Then redirects internally using `targetport` which might be `4040` for instance. Hence `port` and `targetport` should be different.

Added sepcification in the CRD:

```
            sparkUIOptions:
              properties:
                ingressAnnotations:
                  additionalProperties:
                    type: string
                  type: object
                ingressTLS:
                  items:
                    properties:
                      hosts:
                        items:
                          type: string
                        type: array
                      secretName:
                        type: string
                    type: object
                  type: array
                servicePort:
                  format: int32
                  type: integer
              required:
              - servicePort
              type: object
```

Example in SparkApplication specification:

```
  sparkUIOptions:(OPTIONAL)
    servicePort: 80            => (Optional)
    ingressAnnotations:    => (OPTIONAL)
      "kubernetes.io/ingress.class": "nginx"
    ingressTLS:                  => (OPTIONAL)
      - hosts:
          - "logs-normalizer.zzz.yyy.xxx.company.com"
        serviceName: ""
```

After deploying the app:

Service:
```
logs-normalizer-ui-svc:

apiVersion: v1
kind: Service
...
spec:
  clusterIP: xxx.yy.zzz.245
  ports:
  - name: spark-driver-ui-port
    port: 80
    protocol: TCP
    targetPort: 4040
  selector:
    spark-role: driver
    sparkoperator.k8s.io/app-name: logs-normalizer
  sessionAffinity: None
  type: ClusterIP
```

Ingress:

```
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
...
...
spec:
  rules:
  - host: logs-normalizer.zzz.yyy.xxx.company.com
    http:
      paths:
      - backend:
          serviceName: logs-normalizer-ui-svc
          servicePort: 80
  tls:
  - hosts:
    - logs-normalizer.zzz.yyy.xxx.company.com
```


Patch:
diff --git a/Dockerfile b/Dockerfile
index 6f11e3f..97393ac 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -17,16 +17,22 @@
 ARG SPARK_IMAGE=gcr.io/spark-operator/spark:v2.4.5
 
 FROM golang:1.14.1-alpine as builder
-ARG DEP_VERSION="0.5.4"
-RUN apk add --no-cache bash git
-ADD https://github.com/golang/dep/releases/download/v${DEP_VERSION}/dep-linux-amd64 /usr/bin/dep
-RUN chmod +x /usr/bin/dep
-
-WORKDIR ${GOPATH}/src/github.com/GoogleCloudPlatform/spark-on-k8s-operator
-COPY Gopkg.toml Gopkg.lock ./
-RUN dep ensure -vendor-only
-COPY . ./
-RUN go generate && CGO_ENABLED=0 GOOS=linux go build -o /usr/bin/spark-operator
+
+WORKDIR /workspace
+
+# Copy the Go Modules manifests
+COPY go.mod go.mod
+COPY go.sum go.sum
+# Cache deps before building and copying source so that we don't need to re-download as much
+# and so that source changes don't invalidate our downloaded layer
+RUN go mod download
+
+# Copy the go source code
+COPY main.go main.go
+COPY pkg/ pkg/
+
+# Build
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o /usr/bin/spark-operator main.go
 
 FROM ${SPARK_IMAGE}
 COPY --from=builder /usr/bin/spark-operator /usr/bin/
diff --git a/Dockerfile.rh b/Dockerfile.rh
index 8598323..75e7325 100644
--- a/Dockerfile.rh
+++ b/Dockerfile.rh
@@ -23,16 +23,22 @@
 ARG SPARK_IMAGE=gcr.io/spark-operator/spark:v2.4.5
 
 FROM golang:1.14.0-alpine as builder
-ARG DEP_VERSION="0.5.4"
-RUN apk add --no-cache bash git 
-ADD https://github.com/golang/dep/releases/download/v${DEP_VERSION}/dep-linux-amd64 /usr/bin/dep
-RUN chmod +x /usr/bin/dep
-
-WORKDIR ${GOPATH}/src/github.com/GoogleCloudPlatform/spark-on-k8s-operator
-COPY Gopkg.toml Gopkg.lock ./
-RUN dep ensure -vendor-only
-COPY . ./
-RUN go generate && CGO_ENABLED=0 GOOS=linux go build -o /usr/bin/spark-operator
+
+WORKDIR /workspace
+
+# Copy the Go Modules manifests
+COPY go.mod go.mod
+COPY go.sum go.sum
+# Cache deps before building and copying source so that we don't need to re-download as much
+# and so that source changes don't invalidate our downloaded layer
+RUN go mod download
+
+# Copy the go source code
+COPY main.go main.go
+COPY pkg/ pkg/
+
+# Build
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o /usr/bin/spark-operator main.go
 
 FROM ${SPARK_IMAGE}
 COPY --from=builder /usr/bin/spark-operator /usr/bin/
diff --git a/docs/api-docs.md b/docs/api-docs.md
index c2794f9..262f08e 100644
--- a/docs/api-docs.md
+++ b/docs/api-docs.md
@@ -572,6 +572,20 @@ BatchSchedulerConfiguration
 <p>BatchSchedulerOptions provides fine-grained control on how to batch scheduling.</p>
 </td>
 </tr>
+<tr>
+<td>
+<code>sparkUIOptions</code></br>
+<em>
+<a href="#sparkoperator.k8s.io/v1beta2.SparkUIConfiguration">
+SparkUIConfiguration
+</a>
+</em>
+</td>
+<td>
+<em>(Optional)</em>
+<p>SparkUIOptions allows configuring the Service and the Ingress to expose the sparkUI</p>
+</td>
+</tr>
 </table>
 </td>
 </tr>
@@ -937,6 +951,11 @@ Kubernetes core/v1.Lifecycle
 </tr>
 </tbody>
 </table>
+<h3 id="sparkoperator.k8s.io/v1beta2.DriverState">DriverState
+(<code>string</code> alias)</p></h3>
+<p>
+<p>DriverState tells the current state of a spark driver.</p>
+</p>
 <h3 id="sparkoperator.k8s.io/v1beta2.ExecutorSpec">ExecutorSpec
 </h3>
 <p>
@@ -1367,8 +1386,7 @@ int64
 </td>
 <td>
 <em>(Optional)</em>
-<p>OnSubmissionFailureRetryInterval is the interval between retries on failed submissions.
-Interval to wait between successive retries of a failed application.</p>
+<p>OnSubmissionFailureRetryInterval is the interval in seconds between retries on failed submissions.</p>
 </td>
 </tr>
 <tr>
@@ -1380,7 +1398,7 @@ int64
 </td>
 <td>
 <em>(Optional)</em>
-<p>OnFailureRetryInterval is the interval between retries on failed runs.</p>
+<p>OnFailureRetryInterval is the interval in seconds between retries on failed runs.</p>
 </td>
 </tr>
 </tbody>
@@ -2027,6 +2045,20 @@ BatchSchedulerConfiguration
 <p>BatchSchedulerOptions provides fine-grained control on how to batch scheduling.</p>
 </td>
 </tr>
+<tr>
+<td>
+<code>sparkUIOptions</code></br>
+<em>
+<a href="#sparkoperator.k8s.io/v1beta2.SparkUIConfiguration">
+SparkUIConfiguration
+</a>
+</em>
+</td>
+<td>
+<em>(Optional)</em>
+<p>SparkUIOptions allows configuring the Service and the Ingress to expose the sparkUI</p>
+</td>
+</tr>
 </tbody>
 </table>
 <h3 id="sparkoperator.k8s.io/v1beta2.SparkApplicationStatus">SparkApplicationStatus
@@ -2518,8 +2550,66 @@ int64
 </tr>
 </tbody>
 </table>
+<h3 id="sparkoperator.k8s.io/v1beta2.SparkUIConfiguration">SparkUIConfiguration
+</h3>
+<p>
+(<em>Appears on:</em>
+<a href="#sparkoperator.k8s.io/v1beta2.SparkApplicationSpec">SparkApplicationSpec</a>)
+</p>
+<p>
+<p>Specific SparkUI config parameters</p>
+</p>
+<table>
+<thead>
+<tr>
+<th>Field</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>
+<code>servicePort</code></br>
+<em>
+int32
+</em>
+</td>
+<td>
+<em>(Optional)</em>
+<p>ServicePort allows configuring the port at service level that might be different from the targetPort.
+TargetPort should be the same as the one defined in spark.ui.port</p>
+</td>
+</tr>
+<tr>
+<td>
+<code>ingressAnnotations</code></br>
+<em>
+map[string]string
+</em>
+</td>
+<td>
+<em>(Optional)</em>
+<p>IngressAnnotations is a map of key,value pairs of annotations that might be added to the ingress object. i.e. specify nginx as ingress.class</p>
+</td>
+</tr>
+<tr>
+<td>
+<code>ingressTLS</code></br>
+<em>
+<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#ingresstls-v1beta1-extensions">
+[]Kubernetes extensions/v1beta1.IngressTLS
+</a>
+</em>
+</td>
+<td>
+<em>(Optional)</em>
+<p>TlsHosts is useful If we need to declare SSL certificates to the ingress object</p>
+</td>
+</tr>
+</tbody>
+</table>
 <hr/>
 <p><em>
 Generated with <code>gen-crd-api-reference-docs</code>
-on git commit <code>8fdcbf2f5</code>.
+on git commit <code>0e7e97d</code>.
 </em></p>
diff --git a/manifest/crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml b/manifest/crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml
index 5048a79..ea6432d 100644
--- a/manifest/crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml
+++ b/manifest/crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml
@@ -3,6 +3,8 @@
 apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: (unknown)
   creationTimestamp: null
   name: scheduledsparkapplications.sparkoperator.k8s.io
 spec:
@@ -412,7 +414,11 @@ spec:
                                   containerName:
                                     type: string
                                   divisor:
-                                    type: string
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                    x-kubernetes-int-or-string: true
                                   resource:
                                     type: string
                                 required:
@@ -531,7 +537,11 @@ spec:
                                         containerName:
                                           type: string
                                         divisor:
-                                          type: string
+                                          anyOf:
+                                          - type: integer
+                                          - type: string
+                                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                          x-kubernetes-int-or-string: true
                                         resource:
                                           type: string
                                       required:
@@ -840,11 +850,19 @@ spec:
                             properties:
                               limits:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                               requests:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                             type: object
                           securityContext:
@@ -887,6 +905,80 @@ spec:
                                   user:
                                     type: string
                                 type: object
+                              windowsOptions:
+                                properties:
+                                  gmsaCredentialSpec:
+                                    type: string
+                                  gmsaCredentialSpecName:
+                                    type: string
+                                  runAsUserName:
+                                    type: string
+                                type: object
+                            type: object
+                          startupProbe:
+                            properties:
+                              exec:
+                                properties:
+                                  command:
+                                    items:
+                                      type: string
+                                    type: array
+                                type: object
+                              failureThreshold:
+                                format: int32
+                                type: integer
+                              httpGet:
+                                properties:
+                                  host:
+                                    type: string
+                                  httpHeaders:
+                                    items:
+                                      properties:
+                                        name:
+                                          type: string
+                                        value:
+                                          type: string
+                                      required:
+                                      - name
+                                      - value
+                                      type: object
+                                    type: array
+                                  path:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                  scheme:
+                                    type: string
+                                required:
+                                - port
+                                type: object
+                              initialDelaySeconds:
+                                format: int32
+                                type: integer
+                              periodSeconds:
+                                format: int32
+                                type: integer
+                              successThreshold:
+                                format: int32
+                                type: integer
+                              tcpSocket:
+                                properties:
+                                  host:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                required:
+                                - port
+                                type: object
+                              timeoutSeconds:
+                                format: int32
+                                type: integer
                             type: object
                           stdin:
                             type: boolean
@@ -923,6 +1015,8 @@ spec:
                                   type: boolean
                                 subPath:
                                   type: string
+                                subPathExpr:
+                                  type: string
                               required:
                               - mountPath
                               - name
@@ -1112,6 +1206,15 @@ spec:
                             - value
                             type: object
                           type: array
+                        windowsOptions:
+                          properties:
+                            gmsaCredentialSpec:
+                              type: string
+                            gmsaCredentialSpecName:
+                              type: string
+                            runAsUserName:
+                              type: string
+                          type: object
                       type: object
                     serviceAccount:
                       type: string
@@ -1160,7 +1263,11 @@ spec:
                                         containerName:
                                           type: string
                                         divisor:
-                                          type: string
+                                          anyOf:
+                                          - type: integer
+                                          - type: string
+                                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                          x-kubernetes-int-or-string: true
                                         resource:
                                           type: string
                                       required:
@@ -1469,11 +1576,19 @@ spec:
                             properties:
                               limits:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                               requests:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                             type: object
                           securityContext:
@@ -1516,6 +1631,80 @@ spec:
                                   user:
                                     type: string
                                 type: object
+                              windowsOptions:
+                                properties:
+                                  gmsaCredentialSpec:
+                                    type: string
+                                  gmsaCredentialSpecName:
+                                    type: string
+                                  runAsUserName:
+                                    type: string
+                                type: object
+                            type: object
+                          startupProbe:
+                            properties:
+                              exec:
+                                properties:
+                                  command:
+                                    items:
+                                      type: string
+                                    type: array
+                                type: object
+                              failureThreshold:
+                                format: int32
+                                type: integer
+                              httpGet:
+                                properties:
+                                  host:
+                                    type: string
+                                  httpHeaders:
+                                    items:
+                                      properties:
+                                        name:
+                                          type: string
+                                        value:
+                                          type: string
+                                      required:
+                                      - name
+                                      - value
+                                      type: object
+                                    type: array
+                                  path:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                  scheme:
+                                    type: string
+                                required:
+                                - port
+                                type: object
+                              initialDelaySeconds:
+                                format: int32
+                                type: integer
+                              periodSeconds:
+                                format: int32
+                                type: integer
+                              successThreshold:
+                                format: int32
+                                type: integer
+                              tcpSocket:
+                                properties:
+                                  host:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                required:
+                                - port
+                                type: object
+                              timeoutSeconds:
+                                format: int32
+                                type: integer
                             type: object
                           stdin:
                             type: boolean
@@ -1552,6 +1741,8 @@ spec:
                                   type: boolean
                                 subPath:
                                   type: string
+                                subPathExpr:
+                                  type: string
                               required:
                               - mountPath
                               - name
@@ -1595,6 +1786,8 @@ spec:
                             type: boolean
                           subPath:
                             type: string
+                          subPathExpr:
+                            type: string
                         required:
                         - mountPath
                         - name
@@ -1945,7 +2138,11 @@ spec:
                                   containerName:
                                     type: string
                                   divisor:
-                                    type: string
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                    x-kubernetes-int-or-string: true
                                   resource:
                                     type: string
                                 required:
@@ -2064,7 +2261,11 @@ spec:
                                         containerName:
                                           type: string
                                         divisor:
-                                          type: string
+                                          anyOf:
+                                          - type: integer
+                                          - type: string
+                                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                          x-kubernetes-int-or-string: true
                                         resource:
                                           type: string
                                       required:
@@ -2373,11 +2574,19 @@ spec:
                             properties:
                               limits:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                               requests:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                             type: object
                           securityContext:
@@ -2420,6 +2629,80 @@ spec:
                                   user:
                                     type: string
                                 type: object
+                              windowsOptions:
+                                properties:
+                                  gmsaCredentialSpec:
+                                    type: string
+                                  gmsaCredentialSpecName:
+                                    type: string
+                                  runAsUserName:
+                                    type: string
+                                type: object
+                            type: object
+                          startupProbe:
+                            properties:
+                              exec:
+                                properties:
+                                  command:
+                                    items:
+                                      type: string
+                                    type: array
+                                type: object
+                              failureThreshold:
+                                format: int32
+                                type: integer
+                              httpGet:
+                                properties:
+                                  host:
+                                    type: string
+                                  httpHeaders:
+                                    items:
+                                      properties:
+                                        name:
+                                          type: string
+                                        value:
+                                          type: string
+                                      required:
+                                      - name
+                                      - value
+                                      type: object
+                                    type: array
+                                  path:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                  scheme:
+                                    type: string
+                                required:
+                                - port
+                                type: object
+                              initialDelaySeconds:
+                                format: int32
+                                type: integer
+                              periodSeconds:
+                                format: int32
+                                type: integer
+                              successThreshold:
+                                format: int32
+                                type: integer
+                              tcpSocket:
+                                properties:
+                                  host:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                required:
+                                - port
+                                type: object
+                              timeoutSeconds:
+                                format: int32
+                                type: integer
                             type: object
                           stdin:
                             type: boolean
@@ -2456,6 +2739,8 @@ spec:
                                   type: boolean
                                 subPath:
                                   type: string
+                                subPathExpr:
+                                  type: string
                               required:
                               - mountPath
                               - name
@@ -2543,6 +2828,15 @@ spec:
                             - value
                             type: object
                           type: array
+                        windowsOptions:
+                          properties:
+                            gmsaCredentialSpec:
+                              type: string
+                            gmsaCredentialSpecName:
+                              type: string
+                            runAsUserName:
+                              type: string
+                          type: object
                       type: object
                     sidecars:
                       items:
@@ -2589,7 +2883,11 @@ spec:
                                         containerName:
                                           type: string
                                         divisor:
-                                          type: string
+                                          anyOf:
+                                          - type: integer
+                                          - type: string
+                                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                          x-kubernetes-int-or-string: true
                                         resource:
                                           type: string
                                       required:
@@ -2898,11 +3196,19 @@ spec:
                             properties:
                               limits:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                               requests:
                                 additionalProperties:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 type: object
                             type: object
                           securityContext:
@@ -2945,6 +3251,80 @@ spec:
                                   user:
                                     type: string
                                 type: object
+                              windowsOptions:
+                                properties:
+                                  gmsaCredentialSpec:
+                                    type: string
+                                  gmsaCredentialSpecName:
+                                    type: string
+                                  runAsUserName:
+                                    type: string
+                                type: object
+                            type: object
+                          startupProbe:
+                            properties:
+                              exec:
+                                properties:
+                                  command:
+                                    items:
+                                      type: string
+                                    type: array
+                                type: object
+                              failureThreshold:
+                                format: int32
+                                type: integer
+                              httpGet:
+                                properties:
+                                  host:
+                                    type: string
+                                  httpHeaders:
+                                    items:
+                                      properties:
+                                        name:
+                                          type: string
+                                        value:
+                                          type: string
+                                      required:
+                                      - name
+                                      - value
+                                      type: object
+                                    type: array
+                                  path:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                  scheme:
+                                    type: string
+                                required:
+                                - port
+                                type: object
+                              initialDelaySeconds:
+                                format: int32
+                                type: integer
+                              periodSeconds:
+                                format: int32
+                                type: integer
+                              successThreshold:
+                                format: int32
+                                type: integer
+                              tcpSocket:
+                                properties:
+                                  host:
+                                    type: string
+                                  port:
+                                    anyOf:
+                                    - type: integer
+                                    - type: string
+                                    x-kubernetes-int-or-string: true
+                                required:
+                                - port
+                                type: object
+                              timeoutSeconds:
+                                format: int32
+                                type: integer
                             type: object
                           stdin:
                             type: boolean
@@ -2981,6 +3361,8 @@ spec:
                                   type: boolean
                                 subPath:
                                   type: string
+                                subPathExpr:
+                                  type: string
                               required:
                               - mountPath
                               - name
@@ -3024,6 +3406,8 @@ spec:
                             type: boolean
                           subPath:
                             type: string
+                          subPathExpr:
+                            type: string
                         required:
                         - mountPath
                         - name
@@ -3131,6 +3515,27 @@ spec:
                   type: object
                 sparkConfigMap:
                   type: string
+                sparkUIOptions:
+                  properties:
+                    ingressAnnotations:
+                      additionalProperties:
+                        type: string
+                      type: object
+                    ingressTLS:
+                      items:
+                        properties:
+                          hosts:
+                            items:
+                              type: string
+                            type: array
+                          secretName:
+                            type: string
+                        type: object
+                      type: array
+                    servicePort:
+                      format: int32
+                      type: integer
+                  type: object
                 sparkVersion:
                   type: string
                 timeToLiveSeconds:
@@ -3253,6 +3658,26 @@ spec:
                           optional:
                             type: boolean
                         type: object
+                      csi:
+                        properties:
+                          driver:
+                            type: string
+                          fsType:
+                            type: string
+                          nodePublishSecretRef:
+                            properties:
+                              name:
+                                type: string
+                            type: object
+                          readOnly:
+                            type: boolean
+                          volumeAttributes:
+                            additionalProperties:
+                              type: string
+                            type: object
+                        required:
+                        - driver
+                        type: object
                       downwardAPI:
                         properties:
                           defaultMode:
@@ -3280,7 +3705,11 @@ spec:
                                     containerName:
                                       type: string
                                     divisor:
-                                      type: string
+                                      anyOf:
+                                      - type: integer
+                                      - type: string
+                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                      x-kubernetes-int-or-string: true
                                     resource:
                                       type: string
                                   required:
@@ -3296,7 +3725,11 @@ spec:
                           medium:
                             type: string
                           sizeLimit:
-                            type: string
+                            anyOf:
+                            - type: integer
+                            - type: string
+                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                            x-kubernetes-int-or-string: true
                         type: object
                       fc:
                         properties:
@@ -3521,7 +3954,11 @@ spec:
                                               containerName:
                                                 type: string
                                               divisor:
-                                                type: string
+                                                anyOf:
+                                                - type: integer
+                                                - type: string
+                                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                                x-kubernetes-int-or-string: true
                                               resource:
                                                 type: string
                                             required:
@@ -3579,6 +4016,8 @@ spec:
                             type: boolean
                           registry:
                             type: string
+                          tenant:
+                            type: string
                           user:
                             type: string
                           volume:
@@ -3750,5 +4189,5 @@ status:
   acceptedNames:
     kind: ""
     plural: ""
-  conditions: []
-  storedVersions: []
+  conditions: null
+  storedVersions: null
diff --git a/manifest/crds/sparkoperator.k8s.io_sparkapplications.yaml b/manifest/crds/sparkoperator.k8s.io_sparkapplications.yaml
index 7f2db2f..69be4ac 100644
--- a/manifest/crds/sparkoperator.k8s.io_sparkapplications.yaml
+++ b/manifest/crds/sparkoperator.k8s.io_sparkapplications.yaml
@@ -3,6 +3,8 @@
 apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: (unknown)
   creationTimestamp: null
   name: sparkapplications.sparkoperator.k8s.io
 spec:
@@ -398,7 +400,11 @@ spec:
                               containerName:
                                 type: string
                               divisor:
-                                type: string
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                x-kubernetes-int-or-string: true
                               resource:
                                 type: string
                             required:
@@ -517,7 +523,11 @@ spec:
                                     containerName:
                                       type: string
                                     divisor:
-                                      type: string
+                                      anyOf:
+                                      - type: integer
+                                      - type: string
+                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                      x-kubernetes-int-or-string: true
                                     resource:
                                       type: string
                                   required:
@@ -826,11 +836,19 @@ spec:
                         properties:
                           limits:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                           requests:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                         type: object
                       securityContext:
@@ -873,6 +891,80 @@ spec:
                               user:
                                 type: string
                             type: object
+                          windowsOptions:
+                            properties:
+                              gmsaCredentialSpec:
+                                type: string
+                              gmsaCredentialSpecName:
+                                type: string
+                              runAsUserName:
+                                type: string
+                            type: object
+                        type: object
+                      startupProbe:
+                        properties:
+                          exec:
+                            properties:
+                              command:
+                                items:
+                                  type: string
+                                type: array
+                            type: object
+                          failureThreshold:
+                            format: int32
+                            type: integer
+                          httpGet:
+                            properties:
+                              host:
+                                type: string
+                              httpHeaders:
+                                items:
+                                  properties:
+                                    name:
+                                      type: string
+                                    value:
+                                      type: string
+                                  required:
+                                  - name
+                                  - value
+                                  type: object
+                                type: array
+                              path:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                              scheme:
+                                type: string
+                            required:
+                            - port
+                            type: object
+                          initialDelaySeconds:
+                            format: int32
+                            type: integer
+                          periodSeconds:
+                            format: int32
+                            type: integer
+                          successThreshold:
+                            format: int32
+                            type: integer
+                          tcpSocket:
+                            properties:
+                              host:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                            required:
+                            - port
+                            type: object
+                          timeoutSeconds:
+                            format: int32
+                            type: integer
                         type: object
                       stdin:
                         type: boolean
@@ -909,6 +1001,8 @@ spec:
                               type: boolean
                             subPath:
                               type: string
+                            subPathExpr:
+                              type: string
                           required:
                           - mountPath
                           - name
@@ -1098,6 +1192,15 @@ spec:
                         - value
                         type: object
                       type: array
+                    windowsOptions:
+                      properties:
+                        gmsaCredentialSpec:
+                          type: string
+                        gmsaCredentialSpecName:
+                          type: string
+                        runAsUserName:
+                          type: string
+                      type: object
                   type: object
                 serviceAccount:
                   type: string
@@ -1146,7 +1249,11 @@ spec:
                                     containerName:
                                       type: string
                                     divisor:
-                                      type: string
+                                      anyOf:
+                                      - type: integer
+                                      - type: string
+                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                      x-kubernetes-int-or-string: true
                                     resource:
                                       type: string
                                   required:
@@ -1455,11 +1562,19 @@ spec:
                         properties:
                           limits:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                           requests:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                         type: object
                       securityContext:
@@ -1502,6 +1617,80 @@ spec:
                               user:
                                 type: string
                             type: object
+                          windowsOptions:
+                            properties:
+                              gmsaCredentialSpec:
+                                type: string
+                              gmsaCredentialSpecName:
+                                type: string
+                              runAsUserName:
+                                type: string
+                            type: object
+                        type: object
+                      startupProbe:
+                        properties:
+                          exec:
+                            properties:
+                              command:
+                                items:
+                                  type: string
+                                type: array
+                            type: object
+                          failureThreshold:
+                            format: int32
+                            type: integer
+                          httpGet:
+                            properties:
+                              host:
+                                type: string
+                              httpHeaders:
+                                items:
+                                  properties:
+                                    name:
+                                      type: string
+                                    value:
+                                      type: string
+                                  required:
+                                  - name
+                                  - value
+                                  type: object
+                                type: array
+                              path:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                              scheme:
+                                type: string
+                            required:
+                            - port
+                            type: object
+                          initialDelaySeconds:
+                            format: int32
+                            type: integer
+                          periodSeconds:
+                            format: int32
+                            type: integer
+                          successThreshold:
+                            format: int32
+                            type: integer
+                          tcpSocket:
+                            properties:
+                              host:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                            required:
+                            - port
+                            type: object
+                          timeoutSeconds:
+                            format: int32
+                            type: integer
                         type: object
                       stdin:
                         type: boolean
@@ -1538,6 +1727,8 @@ spec:
                               type: boolean
                             subPath:
                               type: string
+                            subPathExpr:
+                              type: string
                           required:
                           - mountPath
                           - name
@@ -1581,6 +1772,8 @@ spec:
                         type: boolean
                       subPath:
                         type: string
+                      subPathExpr:
+                        type: string
                     required:
                     - mountPath
                     - name
@@ -1931,7 +2124,11 @@ spec:
                               containerName:
                                 type: string
                               divisor:
-                                type: string
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                x-kubernetes-int-or-string: true
                               resource:
                                 type: string
                             required:
@@ -2050,7 +2247,11 @@ spec:
                                     containerName:
                                       type: string
                                     divisor:
-                                      type: string
+                                      anyOf:
+                                      - type: integer
+                                      - type: string
+                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                      x-kubernetes-int-or-string: true
                                     resource:
                                       type: string
                                   required:
@@ -2359,11 +2560,19 @@ spec:
                         properties:
                           limits:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                           requests:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                         type: object
                       securityContext:
@@ -2406,6 +2615,80 @@ spec:
                               user:
                                 type: string
                             type: object
+                          windowsOptions:
+                            properties:
+                              gmsaCredentialSpec:
+                                type: string
+                              gmsaCredentialSpecName:
+                                type: string
+                              runAsUserName:
+                                type: string
+                            type: object
+                        type: object
+                      startupProbe:
+                        properties:
+                          exec:
+                            properties:
+                              command:
+                                items:
+                                  type: string
+                                type: array
+                            type: object
+                          failureThreshold:
+                            format: int32
+                            type: integer
+                          httpGet:
+                            properties:
+                              host:
+                                type: string
+                              httpHeaders:
+                                items:
+                                  properties:
+                                    name:
+                                      type: string
+                                    value:
+                                      type: string
+                                  required:
+                                  - name
+                                  - value
+                                  type: object
+                                type: array
+                              path:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                              scheme:
+                                type: string
+                            required:
+                            - port
+                            type: object
+                          initialDelaySeconds:
+                            format: int32
+                            type: integer
+                          periodSeconds:
+                            format: int32
+                            type: integer
+                          successThreshold:
+                            format: int32
+                            type: integer
+                          tcpSocket:
+                            properties:
+                              host:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                            required:
+                            - port
+                            type: object
+                          timeoutSeconds:
+                            format: int32
+                            type: integer
                         type: object
                       stdin:
                         type: boolean
@@ -2442,6 +2725,8 @@ spec:
                               type: boolean
                             subPath:
                               type: string
+                            subPathExpr:
+                              type: string
                           required:
                           - mountPath
                           - name
@@ -2529,6 +2814,15 @@ spec:
                         - value
                         type: object
                       type: array
+                    windowsOptions:
+                      properties:
+                        gmsaCredentialSpec:
+                          type: string
+                        gmsaCredentialSpecName:
+                          type: string
+                        runAsUserName:
+                          type: string
+                      type: object
                   type: object
                 sidecars:
                   items:
@@ -2575,7 +2869,11 @@ spec:
                                     containerName:
                                       type: string
                                     divisor:
-                                      type: string
+                                      anyOf:
+                                      - type: integer
+                                      - type: string
+                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                      x-kubernetes-int-or-string: true
                                     resource:
                                       type: string
                                   required:
@@ -2884,11 +3182,19 @@ spec:
                         properties:
                           limits:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                           requests:
                             additionalProperties:
-                              type: string
+                              anyOf:
+                              - type: integer
+                              - type: string
+                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                              x-kubernetes-int-or-string: true
                             type: object
                         type: object
                       securityContext:
@@ -2931,6 +3237,80 @@ spec:
                               user:
                                 type: string
                             type: object
+                          windowsOptions:
+                            properties:
+                              gmsaCredentialSpec:
+                                type: string
+                              gmsaCredentialSpecName:
+                                type: string
+                              runAsUserName:
+                                type: string
+                            type: object
+                        type: object
+                      startupProbe:
+                        properties:
+                          exec:
+                            properties:
+                              command:
+                                items:
+                                  type: string
+                                type: array
+                            type: object
+                          failureThreshold:
+                            format: int32
+                            type: integer
+                          httpGet:
+                            properties:
+                              host:
+                                type: string
+                              httpHeaders:
+                                items:
+                                  properties:
+                                    name:
+                                      type: string
+                                    value:
+                                      type: string
+                                  required:
+                                  - name
+                                  - value
+                                  type: object
+                                type: array
+                              path:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                              scheme:
+                                type: string
+                            required:
+                            - port
+                            type: object
+                          initialDelaySeconds:
+                            format: int32
+                            type: integer
+                          periodSeconds:
+                            format: int32
+                            type: integer
+                          successThreshold:
+                            format: int32
+                            type: integer
+                          tcpSocket:
+                            properties:
+                              host:
+                                type: string
+                              port:
+                                anyOf:
+                                - type: integer
+                                - type: string
+                                x-kubernetes-int-or-string: true
+                            required:
+                            - port
+                            type: object
+                          timeoutSeconds:
+                            format: int32
+                            type: integer
                         type: object
                       stdin:
                         type: boolean
@@ -2967,6 +3347,8 @@ spec:
                               type: boolean
                             subPath:
                               type: string
+                            subPathExpr:
+                              type: string
                           required:
                           - mountPath
                           - name
@@ -3010,6 +3392,8 @@ spec:
                         type: boolean
                       subPath:
                         type: string
+                      subPathExpr:
+                        type: string
                     required:
                     - mountPath
                     - name
@@ -3117,6 +3501,27 @@ spec:
               type: object
             sparkConfigMap:
               type: string
+            sparkUIOptions:
+              properties:
+                ingressAnnotations:
+                  additionalProperties:
+                    type: string
+                  type: object
+                ingressTLS:
+                  items:
+                    properties:
+                      hosts:
+                        items:
+                          type: string
+                        type: array
+                      secretName:
+                        type: string
+                    type: object
+                  type: array
+                servicePort:
+                  format: int32
+                  type: integer
+              type: object
             sparkVersion:
               type: string
             timeToLiveSeconds:
@@ -3239,6 +3644,26 @@ spec:
                       optional:
                         type: boolean
                     type: object
+                  csi:
+                    properties:
+                      driver:
+                        type: string
+                      fsType:
+                        type: string
+                      nodePublishSecretRef:
+                        properties:
+                          name:
+                            type: string
+                        type: object
+                      readOnly:
+                        type: boolean
+                      volumeAttributes:
+                        additionalProperties:
+                          type: string
+                        type: object
+                    required:
+                    - driver
+                    type: object
                   downwardAPI:
                     properties:
                       defaultMode:
@@ -3266,7 +3691,11 @@ spec:
                                 containerName:
                                   type: string
                                 divisor:
-                                  type: string
+                                  anyOf:
+                                  - type: integer
+                                  - type: string
+                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                  x-kubernetes-int-or-string: true
                                 resource:
                                   type: string
                               required:
@@ -3282,7 +3711,11 @@ spec:
                       medium:
                         type: string
                       sizeLimit:
-                        type: string
+                        anyOf:
+                        - type: integer
+                        - type: string
+                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                        x-kubernetes-int-or-string: true
                     type: object
                   fc:
                     properties:
@@ -3507,7 +3940,11 @@ spec:
                                           containerName:
                                             type: string
                                           divisor:
-                                            type: string
+                                            anyOf:
+                                            - type: integer
+                                            - type: string
+                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+                                            x-kubernetes-int-or-string: true
                                           resource:
                                             type: string
                                         required:
@@ -3565,6 +4002,8 @@ spec:
                         type: boolean
                       registry:
                         type: string
+                      tenant:
+                        type: string
                       user:
                         type: string
                       volume:
@@ -3759,5 +4198,5 @@ status:
   acceptedNames:
     kind: ""
     plural: ""
-  conditions: []
-  storedVersions: []
+  conditions: null
+  storedVersions: null
diff --git a/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go b/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go
index 8ee8337..c470820 100644
--- a/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go
+++ b/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go
@@ -18,6 +18,7 @@ package v1beta2
 
 import (
 	apiv1 "k8s.io/api/core/v1"
+	extensions "k8s.io/api/extensions/v1beta1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
@@ -272,6 +273,9 @@ type SparkApplicationSpec struct {
 	// BatchSchedulerOptions provides fine-grained control on how to batch scheduling.
 	// +optional
 	BatchSchedulerOptions *BatchSchedulerConfiguration `json:"batchSchedulerOptions,omitempty"`
+	// SparkUIOptions allows configuring the Service and the Ingress to expose the sparkUI
+	// +optional
+	SparkUIOptions *SparkUIConfiguration `json:"sparkUIOptions,omitempty"`
 }
 
 // BatchSchedulerConfiguration used to configure how to batch scheduling Spark Application
@@ -284,6 +288,20 @@ type BatchSchedulerConfiguration struct {
 	PriorityClassName *string `json:"priorityClassName,omitempty"`
 }
 
+// Specific SparkUI config parameters
+type SparkUIConfiguration struct {
+	// ServicePort allows configuring the port at service level that might be different from the targetPort.
+	// TargetPort should be the same as the one defined in spark.ui.port
+	// +optional
+	ServicePort *int32 `json:"servicePort"`
+	// IngressAnnotations is a map of key,value pairs of annotations that might be added to the ingress object. i.e. specify nginx as ingress.class
+	// +optional
+	IngressAnnotations map[string]string `json:"ingressAnnotations,omitempty"`
+	// TlsHosts is useful If we need to declare SSL certificates to the ingress object
+	// +optional
+	IngressTLS []extensions.IngressTLS `json:"ingressTLS,omitempty"`
+}
+
 // ApplicationStateType represents the type of the current state of an application.
 type ApplicationStateType string
 
diff --git a/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go b/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go
index 67b2d66..4dda37f 100644
--- a/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go
+++ b/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go
@@ -24,6 +24,7 @@ package v1beta2
 
 import (
 	v1 "k8s.io/api/core/v1"
+	v1beta1 "k8s.io/api/extensions/v1beta1"
 	runtime "k8s.io/apimachinery/pkg/runtime"
 )
 
@@ -654,6 +655,11 @@ func (in *SparkApplicationSpec) DeepCopyInto(out *SparkApplicationSpec) {
 		*out = new(BatchSchedulerConfiguration)
 		(*in).DeepCopyInto(*out)
 	}
+	if in.SparkUIOptions != nil {
+		in, out := &in.SparkUIOptions, &out.SparkUIOptions
+		*out = new(SparkUIConfiguration)
+		(*in).DeepCopyInto(*out)
+	}
 	return
 }
 
@@ -856,3 +862,38 @@ func (in *SparkPodSpec) DeepCopy() *SparkPodSpec {
 	in.DeepCopyInto(out)
 	return out
 }
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *SparkUIConfiguration) DeepCopyInto(out *SparkUIConfiguration) {
+	*out = *in
+	if in.ServicePort != nil {
+		in, out := &in.ServicePort, &out.ServicePort
+		*out = new(int32)
+		**out = **in
+	}
+	if in.IngressAnnotations != nil {
+		in, out := &in.IngressAnnotations, &out.IngressAnnotations
+		*out = make(map[string]string, len(*in))
+		for key, val := range *in {
+			(*out)[key] = val
+		}
+	}
+	if in.IngressTLS != nil {
+		in, out := &in.IngressTLS, &out.IngressTLS
+		*out = make([]v1beta1.IngressTLS, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
+	return
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SparkUIConfiguration.
+func (in *SparkUIConfiguration) DeepCopy() *SparkUIConfiguration {
+	if in == nil {
+		return nil
+	}
+	out := new(SparkUIConfiguration)
+	in.DeepCopyInto(out)
+	return out
+}
diff --git a/pkg/controller/sparkapplication/sparkapp_util.go b/pkg/controller/sparkapplication/sparkapp_util.go
index 7c19d31..7808ecf 100644
--- a/pkg/controller/sparkapplication/sparkapp_util.go
+++ b/pkg/controller/sparkapplication/sparkapp_util.go
@@ -26,6 +26,7 @@ import (
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta2"
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/config"
 	apiv1 "k8s.io/api/core/v1"
+	"k8s.io/api/extensions/v1beta1"
 )
 
 // Helper method to create a key with namespace and appName
@@ -72,6 +73,26 @@ func getResourceLabels(app *v1beta2.SparkApplication) map[string]string {
 	return labels
 }
 
+func getIngressResourceAnnotations(app *v1beta2.SparkApplication) map[string]string {
+	ingressAnnotations := map[string]string{}
+	if app.Spec.SparkUIOptions != nil && app.Spec.SparkUIOptions.IngressAnnotations != nil {
+		for key, value := range app.Spec.SparkUIOptions.IngressAnnotations {
+			ingressAnnotations[key] = value
+		}
+	}
+	return ingressAnnotations
+}
+
+func getIngressTlsHosts(app *v1beta2.SparkApplication) []v1beta1.IngressTLS {
+	var ingressTls []v1beta1.IngressTLS
+	if app.Spec.SparkUIOptions != nil && app.Spec.SparkUIOptions.IngressTLS != nil {
+		for _, ingTls := range app.Spec.SparkUIOptions.IngressTLS {
+			ingressTls = append(ingressTls, ingTls)
+		}
+	}
+	return ingressTls
+}
+
 func podPhaseToExecutorState(podPhase apiv1.PodPhase) v1beta2.ExecutorState {
 	switch podPhase {
 	case apiv1.PodPending:
diff --git a/pkg/controller/sparkapplication/sparkui.go b/pkg/controller/sparkapplication/sparkui.go
index c58dffa..9aeb014 100644
--- a/pkg/controller/sparkapplication/sparkui.go
+++ b/pkg/controller/sparkapplication/sparkui.go
@@ -34,8 +34,8 @@ import (
 )
 
 const (
-	sparkUIPortConfigurationKey = "spark.ui.port"
-	defaultSparkWebUIPort       = "4040"
+	sparkUIPortConfigurationKey       = "spark.ui.port"
+	defaultSparkWebUIPort       int32 = 4040
 )
 
 var ingressURLRegex = regexp.MustCompile("{{\\s*[$]appName\\s*}}")
@@ -48,6 +48,7 @@ func getSparkUIingressURL(ingressURLFormat string, appName string) string {
 type SparkService struct {
 	serviceName string
 	servicePort int32
+	targetPort  intstr.IntOrString
 	serviceIP   string
 }
 
@@ -55,10 +56,14 @@ type SparkService struct {
 type SparkIngress struct {
 	ingressName string
 	ingressURL  string
+	annotations map[string]string
+	ingressTLS  []extensions.IngressTLS
 }
 
 func createSparkUIIngress(app *v1beta2.SparkApplication, service SparkService, ingressURLFormat string, kubeClient clientset.Interface) (*SparkIngress, error) {
 	ingressURL := getSparkUIingressURL(ingressURLFormat, app.GetName())
+	ingressResourceAnnotations := getIngressResourceAnnotations(app)
+	ingressTlsHosts := getIngressTlsHosts(app)
 	ingress := extensions.Ingress{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:            getDefaultUIIngressName(app),
@@ -85,6 +90,13 @@ func createSparkUIIngress(app *v1beta2.SparkApplication, service SparkService, i
 			}},
 		},
 	}
+
+	if len(ingressResourceAnnotations) != 0 {
+		ingress.ObjectMeta.Annotations = ingressResourceAnnotations
+	}
+	if len(ingressTlsHosts) != 0 {
+		ingress.Spec.TLS = ingressTlsHosts
+	}
 	glog.Infof("Creating an Ingress %s for the Spark UI for application %s", ingress.Name, app.Name)
 	_, err := kubeClient.ExtensionsV1beta1().Ingresses(ingress.Namespace).Create(&ingress)
 
@@ -94,18 +106,22 @@ func createSparkUIIngress(app *v1beta2.SparkApplication, service SparkService, i
 	return &SparkIngress{
 		ingressName: ingress.Name,
 		ingressURL:  ingressURL,
+		annotations: ingress.Annotations,
+		ingressTLS:  ingress.Spec.TLS,
 	}, nil
 }
 
 func createSparkUIService(
 	app *v1beta2.SparkApplication,
 	kubeClient clientset.Interface) (*SparkService, error) {
-	portStr := getUITargetPort(app)
-	port, err := strconv.Atoi(portStr)
+	port, err := getUIServicePort(app)
 	if err != nil {
-		return nil, fmt.Errorf("invalid Spark UI port: %s", portStr)
+		return nil, fmt.Errorf("invalid Spark UI servicePort: %d", port)
+	}
+	tPort, err := getUITargetPort(app)
+	if err != nil {
+		return nil, fmt.Errorf("invalid Spark UI targetPort: %d", tPort)
 	}
-
 	service := &apiv1.Service{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:            getDefaultUIServiceName(app),
@@ -117,7 +133,11 @@ func createSparkUIService(
 			Ports: []apiv1.ServicePort{
 				{
 					Name: "spark-driver-ui-port",
-					Port: int32(port),
+					Port: port,
+					TargetPort: intstr.IntOrString{
+						Type:   intstr.Int,
+						IntVal: tPort,
+					},
 				},
 			},
 			Selector: map[string]string{
@@ -137,6 +157,7 @@ func createSparkUIService(
 	return &SparkService{
 		serviceName: service.Name,
 		servicePort: service.Spec.Ports[0].Port,
+		targetPort:  service.Spec.Ports[0].TargetPort,
 		serviceIP:   service.Spec.ClusterIP,
 	}, nil
 }
@@ -144,10 +165,23 @@ func createSparkUIService(
 // getWebUITargetPort attempts to get the Spark web UI port from configuration property spark.ui.port
 // in Spec.SparkConf if it is present, otherwise the default port is returned.
 // Note that we don't attempt to get the port from Spec.SparkConfigMap.
-func getUITargetPort(app *v1beta2.SparkApplication) string {
-	port, ok := app.Spec.SparkConf[sparkUIPortConfigurationKey]
+func getUITargetPort(app *v1beta2.SparkApplication) (int32, error) {
+	portStr, ok := app.Spec.SparkConf[sparkUIPortConfigurationKey]
 	if ok {
-		return port
+		port, err := strconv.Atoi(portStr)
+		return int32(port), err
+	}
+	return defaultSparkWebUIPort, nil
+}
+
+func getUIServicePort(app *v1beta2.SparkApplication) (int32, error) {
+
+	if app.Spec.SparkUIOptions == nil {
+		return getUITargetPort(app)
+	}
+	port := app.Spec.SparkUIOptions.ServicePort
+	if port != nil {
+		return *port, nil
 	}
-	return defaultSparkWebUIPort
+	return defaultSparkWebUIPort, nil
 }


Test patch:
diff --git a/pkg/controller/sparkapplication/sparkui_test.go b/pkg/controller/sparkapplication/sparkui_test.go
index 915a2c9..643d813 100644
--- a/pkg/controller/sparkapplication/sparkui_test.go
+++ b/pkg/controller/sparkapplication/sparkui_test.go
@@ -19,11 +19,12 @@ package sparkapplication
 import (
 	"fmt"
 	"reflect"
-	"strconv"
 	"testing"
 
 	apiv1 "k8s.io/api/core/v1"
+	extensions "k8s.io/api/extensions/v1beta1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/util/intstr"
 	"k8s.io/client-go/kubernetes/fake"
 
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta2"
@@ -47,11 +48,9 @@ func TestCreateSparkUIService(t *testing.T) {
 			}
 			t.Fatal(err)
 		}
-
 		if sparkService.serviceName != test.expectedService.serviceName {
 			t.Errorf("%s: for service name wanted %s got %s", test.name, test.expectedService.serviceName, sparkService.serviceName)
 		}
-
 		service, err := fakeClient.CoreV1().
 			Services(test.app.Namespace).
 			Get(sparkService.serviceName, metav1.GetOptions{})
@@ -78,12 +77,7 @@ func TestCreateSparkUIService(t *testing.T) {
 			t.Errorf("%s: unexpected port wanted %d got %d", test.name, test.expectedService.servicePort, port.Port)
 		}
 	}
-
-	defaultPort, err := strconv.Atoi(defaultSparkWebUIPort)
-	if err != nil {
-		t.Fatal(err)
-	}
-
+	defaultPort := defaultSparkWebUIPort
 	app1 := &v1beta2.SparkApplication{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      "foo",
@@ -126,13 +120,38 @@ func TestCreateSparkUIService(t *testing.T) {
 			SparkApplicationID: "foo-3",
 		},
 	}
+	var appPort int32 = 80
+	app4 := &v1beta2.SparkApplication{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "foo",
+			Namespace: "default",
+			UID:       "foo-123",
+		},
+		Spec: v1beta2.SparkApplicationSpec{
+			SparkUIOptions: &v1beta2.SparkUIConfiguration{
+				ServicePort:        &appPort,
+				IngressAnnotations: nil,
+				IngressTLS:         nil,
+			},
+			SparkConf: map[string]string{
+				sparkUIPortConfigurationKey: "4041",
+			},
+		},
+		Status: v1beta2.SparkApplicationStatus{
+			SparkApplicationID: "foo-3",
+		},
+	}
 	testcases := []testcase{
 		{
-			name: "service with custom port",
+			name: "service with custom serviceport and serviceport and target port are same",
 			app:  app1,
 			expectedService: SparkService{
 				serviceName: fmt.Sprintf("%s-ui-svc", app1.GetName()),
 				servicePort: 4041,
+				targetPort: intstr.IntOrString{
+					Type:   intstr.Int,
+					IntVal: int32(4041),
+				},
 			},
 			expectedSelector: map[string]string{
 				config.SparkAppNameLabel: "foo",
@@ -153,6 +172,23 @@ func TestCreateSparkUIService(t *testing.T) {
 			},
 			expectError: false,
 		},
+		{
+			name: "service with custom serviceport and serviceport and target port are different",
+			app:  app4,
+			expectedService: SparkService{
+				serviceName: fmt.Sprintf("%s-ui-svc", app4.GetName()),
+				servicePort: 80,
+				targetPort: intstr.IntOrString{
+					Type:   intstr.Int,
+					IntVal: int32(4041),
+				},
+			},
+			expectedSelector: map[string]string{
+				config.SparkAppNameLabel: "foo",
+				config.SparkRoleLabel:    config.SparkDriverRole,
+			},
+			expectError: false,
+		},
 		{
 			name:        "service with bad port configurations",
 			app:         app3,
@@ -165,8 +201,78 @@ func TestCreateSparkUIService(t *testing.T) {
 }
 
 func TestCreateSparkUIIngress(t *testing.T) {
+	type testcase struct {
+		name            string
+		app             *v1beta2.SparkApplication
+		expectedIngress SparkIngress
+		expectError     bool
+	}
+
+	testFn := func(test testcase, t *testing.T) {
+		fakeClient := fake.NewSimpleClientset()
+		ingressFormat := "{{$appName}}.ingress.clusterName.com"
+		sparkService, err := createSparkUIService(test.app, fakeClient)
+		sparkIngress, err := createSparkUIIngress(test.app, *sparkService, ingressFormat, fakeClient)
+		if err != nil {
+			if test.expectError {
+				return
+			}
+			t.Fatal(err)
+		}
+		if sparkIngress.ingressName != test.expectedIngress.ingressName {
+			t.Errorf("Ingress name wanted %s got %s", test.expectedIngress.ingressName, sparkIngress.ingressName)
+		}
+		if sparkIngress.ingressURL != test.expectedIngress.ingressURL {
+			t.Errorf("Ingress name wanted %s got %s", test.expectedIngress.ingressURL, sparkIngress.ingressURL)
+		}
+		ingress, err := fakeClient.ExtensionsV1beta1().Ingresses(test.app.Namespace).
+			Get(sparkIngress.ingressName, metav1.GetOptions{})
+		if err != nil {
+			t.Fatal(err)
+		}
+		if len(ingress.Annotations) != 0 {
+			for key, value := range ingress.Annotations {
+				if test.expectedIngress.annotations[key] != ingress.Annotations[key] {
+					t.Errorf("Expected annotation: %s=%s but found : %s=%s", key, value, key, ingress.Annotations[key])
+				}
+			}
+		}
+		if len(ingress.Spec.TLS) != 0 {
+			for _, ingressTls := range ingress.Spec.TLS {
+				if ingressTls.Hosts[0] != test.expectedIngress.ingressTLS[0].Hosts[0] {
+					t.Errorf("Expected ingressTls host: %s but found : %s", test.expectedIngress.ingressTLS[0].Hosts[0], ingressTls.Hosts[0])
+				}
+				if ingressTls.SecretName != test.expectedIngress.ingressTLS[0].SecretName {
+					t.Errorf("Expected ingressTls secretName: %s but found : %s", test.expectedIngress.ingressTLS[0].SecretName, ingressTls.SecretName)
+				}
+			}
+		}
+		if ingress.Labels[config.SparkAppNameLabel] != test.app.Name {
+			t.Errorf("Ingress of app %s has the wrong labels", test.app.Name)
+		}
 
-	app := &v1beta2.SparkApplication{
+		if len(ingress.Spec.Rules) != 1 {
+			t.Errorf("No Ingress rules found.")
+		}
+		ingressRule := ingress.Spec.Rules[0]
+		if ingressRule.Host != test.expectedIngress.ingressURL {
+			t.Errorf("Ingress of app %s has the wrong host %s", test.expectedIngress.ingressURL, ingressRule.Host)
+		}
+
+		if len(ingressRule.IngressRuleValue.HTTP.Paths) != 1 {
+			t.Errorf("No Ingress paths found.")
+		}
+		ingressPath := ingressRule.IngressRuleValue.HTTP.Paths[0]
+		if ingressPath.Backend.ServiceName != sparkService.serviceName {
+			t.Errorf("Service name wanted %s got %s", sparkService.serviceName, ingressPath.Backend.ServiceName)
+		}
+		if ingressPath.Backend.ServicePort.IntVal != sparkService.servicePort {
+			t.Errorf("Service port wanted %v got %v", sparkService.servicePort, ingressPath.Backend.ServicePort)
+		}
+	}
+
+	var appPort int32 = 80
+	app1 := &v1beta2.SparkApplication{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      "foo",
 			Namespace: "default",
@@ -179,56 +285,134 @@ func TestCreateSparkUIIngress(t *testing.T) {
 			},
 		},
 	}
-
-	service := SparkService{
-		serviceName: app.GetName() + "-ui-svc",
-		servicePort: 4041,
-	}
-	ingressFormat := "{{$appName}}.ingress.clusterName.com"
-
-	expectedIngress := SparkIngress{
-		ingressName: fmt.Sprintf("%s-ui-ingress", app.GetName()),
-		ingressURL:  app.GetName() + ".ingress.clusterName.com",
-	}
-	fakeClient := fake.NewSimpleClientset()
-	sparkIngress, err := createSparkUIIngress(app, service, ingressFormat, fakeClient)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	if sparkIngress.ingressName != expectedIngress.ingressName {
-		t.Errorf("Ingress name wanted %s got %s", expectedIngress.ingressName, sparkIngress.ingressName)
-	}
-	if sparkIngress.ingressURL != expectedIngress.ingressURL {
-		t.Errorf("Ingress name wanted %s got %s", expectedIngress.ingressURL, sparkIngress.ingressURL)
-	}
-
-	ingress, err := fakeClient.ExtensionsV1beta1().Ingresses(app.Namespace).
-		Get(sparkIngress.ingressName, metav1.GetOptions{})
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	if ingress.Labels[config.SparkAppNameLabel] != app.Name {
-		t.Errorf("Ingress of app %s has the wrong labels", app.Name)
-	}
-
-	if len(ingress.Spec.Rules) != 1 {
-		t.Errorf("No Ingress rules found.")
+	app2 := &v1beta2.SparkApplication{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "foo",
+			Namespace: "default",
+			UID:       "foo-123",
+		},
+		Spec: v1beta2.SparkApplicationSpec{
+			SparkUIOptions: &v1beta2.SparkUIConfiguration{
+				ServicePort: &appPort,
+				IngressAnnotations: map[string]string{
+					"kubernetes.io/ingress.class":                    "nginx",
+					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
+				},
+			},
+		},
+		Status: v1beta2.SparkApplicationStatus{
+			SparkApplicationID: "foo-1",
+			DriverInfo: v1beta2.DriverInfo{
+				WebUIServiceName: "blah-service",
+			},
+		},
 	}
-	ingressRule := ingress.Spec.Rules[0]
-	if ingressRule.Host != expectedIngress.ingressURL {
-		t.Errorf("Ingress of app %s has the wrong host %s", expectedIngress.ingressURL, ingressRule.Host)
+	app3 := &v1beta2.SparkApplication{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "foo",
+			Namespace: "default",
+			UID:       "foo-123",
+		},
+		Spec: v1beta2.SparkApplicationSpec{
+			SparkUIOptions: &v1beta2.SparkUIConfiguration{
+				ServicePort: &appPort,
+				IngressAnnotations: map[string]string{
+					"kubernetes.io/ingress.class":                    "nginx",
+					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
+				},
+				IngressTLS: []extensions.IngressTLS{
+					{[]string{"host1", "host2"}, "secret"},
+				},
+			},
+		},
+		Status: v1beta2.SparkApplicationStatus{
+			SparkApplicationID: "foo-1",
+			DriverInfo: v1beta2.DriverInfo{
+				WebUIServiceName: "blah-service",
+			},
+		},
 	}
-
-	if len(ingressRule.IngressRuleValue.HTTP.Paths) != 1 {
-		t.Errorf("No Ingress paths found.")
+	app4 := &v1beta2.SparkApplication{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "foo",
+			Namespace: "default",
+			UID:       "foo-123",
+		},
+		Spec: v1beta2.SparkApplicationSpec{
+			SparkUIOptions: &v1beta2.SparkUIConfiguration{
+				ServicePort: &appPort,
+				IngressAnnotations: map[string]string{
+					"kubernetes.io/ingress.class": "nginx",
+				},
+				IngressTLS: []extensions.IngressTLS{
+					{[]string{"host1", "host2"}, ""},
+				},
+			},
+		},
+		Status: v1beta2.SparkApplicationStatus{
+			SparkApplicationID: "foo-1",
+			DriverInfo: v1beta2.DriverInfo{
+				WebUIServiceName: "blah-service",
+			},
+		},
 	}
-	ingressPath := ingressRule.IngressRuleValue.HTTP.Paths[0]
-	if ingressPath.Backend.ServiceName != service.serviceName {
-		t.Errorf("Service name wanted %s got %s", service.serviceName, ingressPath.Backend.ServiceName)
+	testcases := []testcase{
+		{
+			name: "simple ingress object",
+			app:  app1,
+			expectedIngress: SparkIngress{
+				ingressName: fmt.Sprintf("%s-ui-ingress", app1.GetName()),
+				ingressURL:  app1.GetName() + ".ingress.clusterName.com",
+			},
+			expectError: false,
+		},
+		{
+			name: "ingress with annotations and without tls configuration",
+			app:  app2,
+			expectedIngress: SparkIngress{
+				ingressName: fmt.Sprintf("%s-ui-ingress", app2.GetName()),
+				ingressURL:  app2.GetName() + ".ingress.clusterName.com",
+				annotations: map[string]string{
+					"kubernetes.io/ingress.class":                    "nginx",
+					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
+				},
+			},
+			expectError: false,
+		},
+		{
+			name: "ingress with annotations and tls configuration",
+			app:  app3,
+			expectedIngress: SparkIngress{
+				ingressName: fmt.Sprintf("%s-ui-ingress", app3.GetName()),
+				ingressURL:  app3.GetName() + ".ingress.clusterName.com",
+				annotations: map[string]string{
+					"kubernetes.io/ingress.class":                    "nginx",
+					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
+				},
+				ingressTLS: []extensions.IngressTLS{
+					{[]string{"host1", "host2"}, "secret"},
+				},
+			},
+			expectError: false,
+		},
+		{
+			name: "ingress with incomplete list of annotations",
+			app:  app4,
+			expectedIngress: SparkIngress{
+				ingressName: fmt.Sprintf("%s-ui-ingress", app4.GetName()),
+				ingressURL:  app3.GetName() + ".ingress.clusterName.com",
+				annotations: map[string]string{
+					"kubernetes.io/ingress.class":                    "nginx",
+					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
+				},
+				ingressTLS: []extensions.IngressTLS{
+					{[]string{"host1", "host2"}, ""},
+				},
+			},
+			expectError: true,
+		},
 	}
-	if ingressPath.Backend.ServicePort.IntVal != service.servicePort {
-		t.Errorf("Service port wanted %v got %v", service.servicePort, ingressPath.Backend.ServicePort)
+	for _, test := range testcases {
+		testFn(test, t)
 	}
 }