Repository: kroxylicious/kroxylicious
Pull request #2058: Add API for `KafkaService`'s `spec.tls`
Author: login: tombentley
Created: 2025-04-15T02:13:07Z | Merged: 2025-04-15T23:11:13Z

### Type of change

- Enhancement / new feature

### Description

* implement ref resolution and validation in the KSR
* don't hook it up to proxy configuration generation yet (this will follow in a later PR).

Part of #1828


Diff:
diff --git a/kroxylicious-operator/pom.xml b/kroxylicious-operator/pom.xml
index 970c2f95dc..ba722db99c 100644
--- a/kroxylicious-operator/pom.xml
+++ b/kroxylicious-operator/pom.xml
@@ -353,6 +353,9 @@
                         <io.kroxylicious.kubernetes.api.v1alpha1.virtualkafkaclusterspec.IngressRefs>
                             io.kroxylicious.kubernetes.api.common.IngressRef
                         </io.kroxylicious.kubernetes.api.v1alpha1.virtualkafkaclusterspec.IngressRefs>
+                        <io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.tls.CertificateRef>
+                            io.kroxylicious.kubernetes.api.common.AnyLocalRef
+                        </io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.tls.CertificateRef>
                         <io.kroxylicious.kubernetes.api.v1alpha1.virtualkafkaclusterspec.FilterRefs>
                             io.kroxylicious.kubernetes.api.common.FilterRef
                         </io.kroxylicious.kubernetes.api.v1alpha1.virtualkafkaclusterspec.FilterRefs>
diff --git a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/AnyLocalRef.java b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/AnyLocalRef.java
index bd6d03078a..bf5fdea542 100644
--- a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/AnyLocalRef.java
+++ b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/AnyLocalRef.java
@@ -9,6 +9,8 @@
 import io.fabric8.kubernetes.api.model.HasMetadata;
 import io.fabric8.kubernetes.api.model.KubernetesResource;
 
+import edu.umd.cs.findbugs.annotations.Nullable;
+
 /**
  * A reference, used in a kubernetes resource, to some kubernetes resource in the same namespace.
  * This is used for references where the kind and group are not known statically.
@@ -44,6 +46,7 @@ public AnyLocalRefBuilder edit() {
     private String group;
 
     @Override
+    @Nullable
     public String getGroup() {
         return group;
     }
@@ -58,6 +61,7 @@ public void setGroup(String group) {
     private String kind;
 
     @Override
+    @Nullable
     public String getKind() {
         return kind;
     }
diff --git a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/Condition.java b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/Condition.java
index 9150ee5cc8..0c70e7f9d6 100644
--- a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/Condition.java
+++ b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/Condition.java
@@ -34,8 +34,10 @@
 public class Condition implements io.fabric8.kubernetes.api.builder.Editable<ConditionBuilder>, io.fabric8.kubernetes.api.model.KubernetesResource {
 
     public static final String REASON_INTERPOLATED_REFS_NOT_FOUND = "InterpolatedReferencedResourcesNotFound";
+    public static final String REASON_REF_GROUP_KIND_NOT_SUPPORTED = "ReferencedGroupKindNotSupported";
     public static final String REASON_REFS_NOT_FOUND = "ReferencedResourcesNotFound";
     public static final String REASON_TRANSITIVE_REFS_NOT_FOUND = "TransitivelyReferencedResourcesNotFound";
+    public static final String REASON_INVALID_REFERENCED_RESOURCE = "InvalidReferencedResource";
     public static final String REASON_INVALID = "Invalid";
 
     @Override
diff --git a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaProtocolFilterReconciler.java b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaProtocolFilterReconciler.java
index f4ec17c2b5..513cbfa1ba 100644
--- a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaProtocolFilterReconciler.java
+++ b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaProtocolFilterReconciler.java
@@ -42,7 +42,7 @@
 
 /**
  * <p>Reconciles a {@link KafkaProtocolFilter} by checking whether the {@link Secret}s
- * and {@link ConfigMap}s refered to in interpolated expressions actually exist, setting a
+ * and {@link ConfigMap}s referred to in interpolated expressions actually exist, setting a
  * {@link Condition.Type#ResolvedRefs} {@link Condition} accordingly.</p>
  *
  * <p>Because a filter CR is not uniquely associated with a {@code KafkaProxy} it's not possible
diff --git a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java
index 1be1c182c1..38bdf1aaf3 100644
--- a/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java
+++ b/kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java
@@ -8,52 +8,200 @@
 
 import java.time.Clock;
 import java.util.List;
+import java.util.Optional;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import io.fabric8.kubernetes.api.model.ConfigMap;
+import io.fabric8.kubernetes.api.model.Secret;
+import io.javaoperatorsdk.operator.api.config.informer.InformerEventSourceConfiguration;
 import io.javaoperatorsdk.operator.api.reconciler.Context;
 import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusUpdateControl;
 import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
 import io.javaoperatorsdk.operator.api.reconciler.UpdateControl;
 import io.javaoperatorsdk.operator.processing.event.source.EventSource;
+import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource;
 
+import io.kroxylicious.kubernetes.api.common.AnyLocalRef;
 import io.kroxylicious.kubernetes.api.common.Condition;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaService;
+import io.kroxylicious.kubernetes.api.v1alpha1.KafkaServiceSpec;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.Tls;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.tls.TrustAnchorRef;
 
+import edu.umd.cs.findbugs.annotations.Nullable;
+
+import static io.kroxylicious.kubernetes.api.common.Condition.Type.ResolvedRefs;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.name;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.namespace;
 
+/**
+ * <p>Reconciles a {@link KafkaService} by checking whether resources referred to in {@code spec.tls.certificateRefs}
+ * actually exist, setting a {@link Condition.Type#ResolvedRefs} {@link Condition} accordingly.</p>
+ *
+ * <p>Because a service CR is not uniquely associated with a {@code KafkaProxy} it's not possible
+ * to set an {@code Accepted} condition on CR instances.</p>
+ */
 public final class KafkaServiceReconciler implements
         io.javaoperatorsdk.operator.api.reconciler.Reconciler<KafkaService> {
 
     private static final Logger LOGGER = LoggerFactory.getLogger(KafkaServiceReconciler.class);
 
+    public static final String SECRETS_EVENT_SOURCE_NAME = "secrets";
+    public static final String CONFIG_MAPS_EVENT_SOURCE_NAME = "configmaps";
+
+    public static AnyLocalRef asRef(TrustAnchorRef reference) {
+        AnyLocalRef result = new AnyLocalRef();
+        result.setGroup(reference.getGroup());
+        result.setKind(reference.getKind());
+        result.setName(reference.getName());
+        return result;
+    }
+
     private final KafkaServiceStatusFactory statusFactory;
 
     public KafkaServiceReconciler(Clock clock) {
         this.statusFactory = new KafkaServiceStatusFactory(clock);
     }
 
+    @Override
+    public List<EventSource<?, KafkaService>> prepareEventSources(EventSourceContext<KafkaService> context) {
+        InformerEventSourceConfiguration<Secret> serviceToSecret = InformerEventSourceConfiguration.from(
+                Secret.class,
+                KafkaService.class)
+                .withName(SECRETS_EVENT_SOURCE_NAME)
+                .withPrimaryToSecondaryMapper((KafkaService cluster) -> ResourcesUtil.localRefAsResourceId(cluster, cluster.getSpec().getTls().getCertificateRef()))
+                .withSecondaryToPrimaryMapper(secret -> ResourcesUtil.findReferrers(context,
+                        secret,
+                        KafkaService.class,
+                        service -> service.getSpec().getTls().getCertificateRef()))
+                .build();
+        InformerEventSourceConfiguration<ConfigMap> serviceToConfigMap = InformerEventSourceConfiguration.from(
+                ConfigMap.class,
+                KafkaService.class)
+                .withName(CONFIG_MAPS_EVENT_SOURCE_NAME)
+                .withPrimaryToSecondaryMapper(
+                        (KafkaService cluster) -> ResourcesUtil.localRefAsResourceId(cluster, asRef(cluster.getSpec().getTls().getTrustAnchorRef())))
+                .withSecondaryToPrimaryMapper(configMap -> ResourcesUtil.findReferrers(context,
+                        configMap,
+                        KafkaService.class,
+                        service -> asRef(service.getSpec().getTls().getTrustAnchorRef())))
+                .build();
+        return List.of(
+                new InformerEventSource<>(serviceToSecret, context),
+                new InformerEventSource<>(serviceToConfigMap, context));
+    }
+
     @Override
     public UpdateControl<KafkaService> reconcile(KafkaService service, Context<KafkaService> context) {
-        UpdateControl<KafkaService> uc = UpdateControl.patchStatus(statusFactory.newTrueConditionStatusPatch(service, Condition.Type.Accepted));
+
+        KafkaService updatedService = null;
+
+        var trustAnchorRefOpt = Optional.ofNullable(service.getSpec())
+                .map(KafkaServiceSpec::getTls)
+                .map(Tls::getTrustAnchorRef);
+        if (trustAnchorRefOpt.isPresent()) {
+            updatedService = checkTrustAnchorRef(service, context, trustAnchorRefOpt.get());
+        }
+
+        if (updatedService == null) {
+            var certRefOpt = Optional.ofNullable(service.getSpec())
+                    .map(KafkaServiceSpec::getTls)
+                    .map(Tls::getCertificateRef);
+            if (certRefOpt.isPresent()) {
+                updatedService = checkCertRef(service, context, certRefOpt.get());
+            }
+        }
+
+        if (updatedService == null) {
+            updatedService = statusFactory.newTrueConditionStatusPatch(service, ResolvedRefs);
+        }
+
+        UpdateControl<KafkaService> uc = UpdateControl.patchStatus(updatedService);
+
         if (LOGGER.isInfoEnabled()) {
             LOGGER.info("Completed reconciliation of {}/{}", namespace(service), name(service));
         }
         return uc;
     }
 
-    @Override
-    public List<EventSource<?, KafkaService>> prepareEventSources(EventSourceContext<KafkaService> context) {
-        return io.javaoperatorsdk.operator.api.reconciler.Reconciler.super.prepareEventSources(context);
+    @Nullable
+    private KafkaService checkTrustAnchorRef(KafkaService service,
+                                             Context<KafkaService> context,
+                                             TrustAnchorRef trustAnchorRef) {
+        String path = "spec.tls.trustAnchorRef";
+        if ("ConfigMap".equals(Optional.ofNullable(trustAnchorRef.getKind()).orElse("ConfigMap"))
+                && Optional.ofNullable(trustAnchorRef.getGroup()).orElse("").isEmpty()) {
+            Optional<ConfigMap> configMapOpt = context.getSecondaryResource(ConfigMap.class, CONFIG_MAPS_EVENT_SOURCE_NAME);
+            if (configMapOpt.isEmpty()) {
+                return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                        Condition.REASON_REFS_NOT_FOUND,
+                        path + ": referenced resource not found");
+            }
+            else {
+                String key = trustAnchorRef.getKey();
+                if (key == null) {
+                    return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                            Condition.REASON_INVALID,
+                            path + " must specify 'key'");
+                }
+                if (!key.endsWith(".pem")
+                        && !key.endsWith(".p12")
+                        && !key.endsWith(".jks")) {
+                    return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                            Condition.REASON_INVALID,
+                            path + ".key should end with .pem, .p12 or .jks");
+                }
+                else if (!configMapOpt.get().getData().containsKey(trustAnchorRef.getKey())) {
+                    return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                            Condition.REASON_INVALID_REFERENCED_RESOURCE,
+                            path + ": referenced resource does not contain key " + trustAnchorRef.getKey());
+                }
+            }
+        }
+        else {
+            return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                    Condition.REASON_REF_GROUP_KIND_NOT_SUPPORTED,
+                    "spec.tls.trustAnchorRef supports referents: configmaps");
+        }
+        return null;
+    }
+
+    @Nullable
+    private KafkaService checkCertRef(KafkaService service,
+                                      Context<KafkaService> context,
+                                      AnyLocalRef certRef) {
+        String path = "spec.tls.certificateRef";
+        if ("Secret".equals(Optional.ofNullable(certRef.getKind()).orElse("Secret"))
+                && Optional.ofNullable(certRef.getGroup()).orElse("").isEmpty()) {
+            Optional<Secret> secretOpt = context.getSecondaryResource(Secret.class, SECRETS_EVENT_SOURCE_NAME);
+            if (secretOpt.isEmpty()) {
+                return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                        Condition.REASON_REFS_NOT_FOUND,
+                        path + ": referenced resource not found");
+            }
+            else {
+                if (!"kubernetes.io/tls".equals(secretOpt.get().getType())) {
+                    return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                            Condition.REASON_INVALID_REFERENCED_RESOURCE,
+                            path + ": referenced secret should have 'type: kubernetes.io/tls'");
+                }
+            }
+        }
+        else {
+            return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                    Condition.REASON_REF_GROUP_KIND_NOT_SUPPORTED,
+                    path + " supports referents: secrets");
+        }
+        return null;
     }
 
     @Override
     public ErrorStatusUpdateControl<KafkaService> updateErrorStatus(KafkaService service, Context<KafkaService> context, Exception e) {
         // ResolvedRefs to UNKNOWN
         ErrorStatusUpdateControl<KafkaService> uc = ErrorStatusUpdateControl
-                .patchStatus(statusFactory.newUnknownConditionStatusPatch(service, Condition.Type.Accepted, e));
+                .patchStatus(statusFactory.newUnknownConditionStatusPatch(service, ResolvedRefs, e));
         if (LOGGER.isInfoEnabled()) {
             LOGGER.info("Completed reconciliation of {}/{} with error {}", namespace(service), name(service), e.toString());
         }
diff --git a/kroxylicious-operator/src/main/resources/META-INF/fabric8/kafkaservices.kroxylicious.io-v1.yml b/kroxylicious-operator/src/main/resources/META-INF/fabric8/kafkaservices.kroxylicious.io-v1.yml
index 04e9cfbbb0..249651a407 100644
--- a/kroxylicious-operator/src/main/resources/META-INF/fabric8/kafkaservices.kroxylicious.io-v1.yml
+++ b/kroxylicious-operator/src/main/resources/META-INF/fabric8/kafkaservices.kroxylicious.io-v1.yml
@@ -44,6 +44,103 @@ spec:
                   type: string
                   minLength: 1
                   pattern: ^(([^:]+:[0-9]{1,5}),)*([^:]+:[0-9]{1,5})$
+                tls:
+                  description: Configuration for TLS connections made to the Kafka cluster
+                  type: object
+                  properties:
+                    certificateRef:
+                      description: |
+                        A reference to a resource holding the TLS client certificate to be used when 
+                        connecting to the Kafka cluster. 
+                        
+                        When the resource is of `kind: Secret` it is expected to have `type: kubernetes.io/tls` with 
+                        keys `tls.key` and `tls.crt`. See https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets.
+                      type: object
+                      required: [ "name" ]
+                      properties:
+                        kind:
+                          description: The API kind of the resource containing the TLS client certificate.
+                          type: string
+                          default: Secret
+                        group:
+                          description: The API group of the resource containing the TLS client certificate.
+                          type: string
+                          default: ""
+                        name:
+                          description: The name of the resource containing the TLS client certificate.
+                          type: string
+                    trustAnchorRef:
+                      description: |
+                        A reference to a resource holding the trusted CA certificates for validation of
+                        a Kafka broker's TLS server certificate.
+                        If not specified the proxy will use the trust store inherited from the platform.
+                      type: object
+                      required: [ "name", "key" ]
+                      properties:
+                        kind:
+                          description: The API kind of the resource containing the TLS client certificate.
+                          type: string
+                          default: ConfigMap
+                        group:
+                          description: The API group of the resource containing the TLS client certificate.
+                          type: string
+                          default: ""
+                        name:
+                          description: The name of the resource containing the TLS client certificate.
+                          type: string
+                        key:
+                          description: | 
+                            The name of the key identifying the certificate bundle within the resource. 
+                            Supported formats for the bundle are: PEM, PKCS#12 and JKS.
+                          type: string
+                    protocols:
+                      description: |
+                        The allowed or denied TLS protocols.
+                        Either `allowed` or `denied` can be given.
+                        The supported protocols are those supported by the proxy JVM.
+                      type: object
+                      minProperties: 1
+                      maxProperties: 1
+                      properties:
+                        allowed:
+                          description: |
+                            The allowed TLS protocols, ordered by preference.
+                            If not specified this defaults to the TLS protocols of the proxy JVM's default SSL context.
+                          type: array
+                          items:
+                            type: string
+                        denied:
+                          description: |
+                            The denied TLS protocols.
+                            If not specified this defaults to the empty list.
+                          type: array
+                          items:
+                            type: string
+                          default: []
+                    cipherSuites:
+                      description: |
+                        The allowed or denied cipher suites.
+                        Either `allowed` or `denied` can be given.
+                        The supported cipher suites are those supported by the proxy JVM.
+                      type: object
+                      minProperties: 1
+                      maxProperties: 1
+                      properties:
+                        allowed:
+                          description: |
+                            The allowed cipher suites.
+                            If not specified a default list of ciphers, which depends on the runtime and enabled protocols, will be used.
+                          type: array
+                          items:
+                            type: string
+                        denied:
+                          description: |
+                            The denied cipher suites.
+                            If not specified this will default to the empty list.
+                          type: array
+                          items:
+                            type: string
+                          default: []
                 nodeIdRanges:
                   type: array
                   minItems: 1
diff --git a/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerIT.java b/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerIT.java
index 44b57d60e8..6fdf7e4d4d 100644
--- a/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerIT.java
+++ b/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerIT.java
@@ -18,10 +18,18 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import io.fabric8.kubernetes.api.model.ConfigMap;
+import io.fabric8.kubernetes.api.model.ConfigMapBuilder;
+import io.fabric8.kubernetes.api.model.Secret;
+import io.fabric8.kubernetes.api.model.SecretBuilder;
 import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension;
 
+import io.kroxylicious.kubernetes.api.common.Condition;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaService;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaServiceBuilder;
+import io.kroxylicious.kubernetes.operator.assertj.KafkaServiceStatusAssert;
+
+import edu.umd.cs.findbugs.annotations.Nullable;
 
 import static io.kroxylicious.kubernetes.operator.assertj.OperatorAssertions.assertThat;
 import static org.awaitility.Awaitility.await;
@@ -30,8 +38,11 @@
 class KafkaServiceReconcilerIT {
 
     private static final Logger LOGGER = LoggerFactory.getLogger(KafkaServiceReconcilerIT.class);
-    private static final String UPDATED_BOOTSTRAP = "bar.bootstrap:9090";
+    public static final String FOO_BOOTSTRAP_9090 = "foo.bootstrap:9090";
+    private static final String BAR_BOOTSTRAP_9090 = "bar.bootstrap:9090";
     public static final String SERVICE_A = "service-a";
+    public static final String SECRET_X = "secret-x";
+    public static final String CONFIG_MAP_T = "configmap-t";
 
     private static final ConditionFactory AWAIT = await().timeout(Duration.ofSeconds(60));
 
@@ -56,54 +67,183 @@ void stopOperator() {
     }
 
     @Test
-    void shouldAcceptKafkaService() {
+    void shouldImmediatelyResolveWhenNoReferents() {
+        // Given
+        KafkaService resource = kafkaService(SERVICE_A, null, null);
+
+        // When
+        testActor.create(resource);
+
+        // Then
+        assertResolvedRefsTrue(resource, FOO_BOOTSTRAP_9090);
+    }
+
+    @Test
+    void shouldResolveUpdateToKafkaService() {
         // Given
-        final KafkaService kafkaService = testActor.create(kafkaService(SERVICE_A));
-        final KafkaService updated = kafkaService.edit().editSpec().withBootstrapServers(UPDATED_BOOTSTRAP).endSpec().build();
+        var kafkaService = testActor.create(new KafkaServiceBuilder().withNewMetadata().withName(SERVICE_A).endMetadata().build());
 
         // When
+        final KafkaService updated = kafkaService.edit().editSpec().withBootstrapServers(BAR_BOOTSTRAP_9090).endSpec().build();
         testActor.replace(updated);
 
         // Then
-        AWAIT.untilAsserted(() -> {
-            final KafkaService mycoolkafkaservice = testActor.get(KafkaService.class, SERVICE_A);
-            Assertions.assertThat(mycoolkafkaservice.getSpec().getBootstrapServers()).isEqualTo(UPDATED_BOOTSTRAP);
-            assertThat(mycoolkafkaservice.getStatus())
-                    .isNotNull()
-                    .conditionList()
-                    .singleElement()
-                    .isAcceptedTrue();
-        });
+        assertResolvedRefsTrue(updated, BAR_BOOTSTRAP_9090);
     }
 
     @Test
-    void shouldAcceptUpdateToKafkaService() {
+    void shouldEventuallyResolveOnceCertSecretCreated() {
         // Given
+        KafkaService resource = kafkaService(SERVICE_A, SECRET_X, null);
 
         // When
-        testActor.create(new KafkaServiceBuilder().withNewMetadata().withName(SERVICE_A).endMetadata().build());
+        final KafkaService kafkaService = testActor.create(resource);
 
         // Then
-        AWAIT.untilAsserted(() -> {
-            final KafkaService mycoolkafkaservice = testActor.get(KafkaService.class, SERVICE_A);
-            assertThat(mycoolkafkaservice.getStatus())
-                    .isNotNull()
-                    .conditionList()
-                    .singleElement()
-                    .isAcceptedTrue();
-        });
+        assertResolvedRefsFalse(kafkaService, Condition.REASON_REFS_NOT_FOUND, "spec.tls.certificateRef: referenced resource not found");
+
+        // And When
+        testActor.create(tlsCertificateSecret(SECRET_X));
+
+        // Then
+        assertResolvedRefsTrue(kafkaService, FOO_BOOTSTRAP_9090);
+
+    }
+
+    @Test
+    void shouldUpdateStatusOnceTlsCertificateSecretDeleted() {
+        // Given
+        var tlsCertSecret = testActor.create(tlsCertificateSecret(SECRET_X));
+        KafkaService resource = testActor.create(kafkaService(SERVICE_A, SECRET_X, null));
+        assertResolvedRefsTrue(resource, FOO_BOOTSTRAP_9090);
+
+        // When
+        testActor.delete(tlsCertSecret);
+
+        // Then
+        assertResolvedRefsFalse(resource, Condition.REASON_REFS_NOT_FOUND, "spec.tls.certificateRef: referenced resource not found");
     }
 
-    private static KafkaService kafkaService(String name) {
+    @Test
+    void shouldEventuallyResolveOnceTrustAnchorCreated() {
+        // Given
+        KafkaService resource = kafkaService(SERVICE_A, null, CONFIG_MAP_T);
+
+        // When
+        final KafkaService kafkaService = testActor.create(resource);
+
+        // Then
+        assertResolvedRefsFalse(kafkaService, Condition.REASON_REFS_NOT_FOUND, "spec.tls.trustAnchorRef: referenced resource not found");
+
+        // And When
+        testActor.create(trustAnchorConfigMap(CONFIG_MAP_T));
+
+        // Then
+        assertResolvedRefsTrue(kafkaService, FOO_BOOTSTRAP_9090);
+
+    }
+
+    @Test
+    void shouldUpdateStatusOnceTrustAnchorDeleted() {
+        // Given
+        var trustedCaCerts = testActor.create(trustAnchorConfigMap(CONFIG_MAP_T));
+        KafkaService resource = testActor.create(kafkaService(SERVICE_A, null, CONFIG_MAP_T));
+        assertResolvedRefsTrue(resource, FOO_BOOTSTRAP_9090);
+
+        // When
+        testActor.delete(trustedCaCerts);
+
+        // Then
+        assertResolvedRefsFalse(resource, Condition.REASON_REFS_NOT_FOUND, "spec.tls.trustAnchorRef: referenced resource not found");
+    }
+
+    private ConfigMap trustAnchorConfigMap(String name) {
+        // @formatter:off
+        return new ConfigMapBuilder()
+                .withNewMetadata()
+                    .withName(name)
+                .endMetadata()
+                .addToData("ca-bundle.pem", "whatever")
+                .build();
+        // @formatter:on
+    }
+
+    private static Secret tlsCertificateSecret(String name) {
         // @formatter:off
-        return new KafkaServiceBuilder()
+        return new SecretBuilder()
+                .withNewMetadata()
+                    .withName(name)
+                .endMetadata()
+                .withType("kubernetes.io/tls")
+                .addToData("tls.key", "whatever")
+                .addToData("tls.crt", "whatever")
+                .build();
+        // @formatter:on
+    }
+
+    private static KafkaService kafkaService(String name, @Nullable String certSecretName, @Nullable String trustResourceName) {
+        // @formatter:off
+        KafkaService result = new KafkaServiceBuilder()
                 .withNewMetadata()
                     .withName(name)
                 .endMetadata()
                 .editOrNewSpec()
-                    .withBootstrapServers("foo.bootstrap:9090")
+                    .withBootstrapServers(FOO_BOOTSTRAP_9090)
                 .endSpec()
+            .build();
+        if (certSecretName != null) {
+            result = new KafkaServiceBuilder(result)
+                    .editSpec()
+                        .editOrNewTls()
+                            .withNewCertificateRef()
+                                .withName(certSecretName)
+                            .endCertificateRef()
+                        .endTls()
+                    .endSpec()
                 .build();
+        }
+        if (trustResourceName != null) {
+            result = new KafkaServiceBuilder(result)
+                    .editSpec()
+                        .editOrNewTls()
+                            .withNewTrustAnchorRef()
+                                .withName(trustResourceName)
+                                .withKey("ca-bundle.pem")
+                            .endTrustAnchorRef()
+                        .endTls()
+                    .endSpec()
+                .build();
+        }
+        return result;
         // @formatter:on
     }
+
+    private void assertResolvedRefsTrue(KafkaService cr, String expectedBootstrap) {
+        AWAIT.untilAsserted(() -> {
+            final KafkaService kafkaService = testActor.get(KafkaService.class, ResourcesUtil.name(cr));
+            Assertions.assertThat(kafkaService).isNotNull();
+            Assertions.assertThat(kafkaService.getSpec().getBootstrapServers()).isEqualTo(expectedBootstrap);
+            assertThat(kafkaService.getStatus())
+                    .isNotNull()
+                    .conditionList()
+                    .singleElement()
+                    .isResolvedRefsTrue();
+        });
+    }
+
+    private void assertResolvedRefsFalse(KafkaService cr,
+                                         String reason,
+                                         String message) {
+        AWAIT.alias("FilterStatusResolvedRefs").untilAsserted(() -> {
+            var kafkaService = testActor.resources(KafkaService.class)
+                    .withName(ResourcesUtil.name(cr)).get();
+            Assertions.assertThat(kafkaService.getStatus()).isNotNull();
+            KafkaServiceStatusAssert
+                    .assertThat(kafkaService.getStatus())
+                    .hasObservedGenerationInSyncWithMetadataOf(kafkaService)
+                    .singleCondition()
+                    .hasObservedGenerationInSyncWithMetadataOf(kafkaService)
+                    .isResolvedRefsFalse(reason, message);
+        });
+    }
 }
diff --git a/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerTest.java b/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerTest.java
index add3048f96..cb23e37655 100644
--- a/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerTest.java
+++ b/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerTest.java
@@ -9,13 +9,23 @@
 import java.time.Clock;
 import java.time.Instant;
 import java.time.ZoneId;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+import java.util.function.Consumer;
 
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
-import org.mockito.Mock;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
 import org.mockito.junit.jupiter.MockitoExtension;
 
+import io.fabric8.kubernetes.api.model.ConfigMap;
+import io.fabric8.kubernetes.api.model.ConfigMapBuilder;
+import io.fabric8.kubernetes.api.model.Secret;
+import io.fabric8.kubernetes.api.model.SecretBuilder;
 import io.fabric8.kubernetes.client.KubernetesClient;
 import io.fabric8.kubernetes.client.server.mock.EnableKubernetesMockClient;
 import io.fabric8.kubernetes.client.server.mock.KubernetesMockServer;
@@ -25,10 +35,13 @@
 import io.kroxylicious.kubernetes.api.common.Condition;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaService;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaServiceBuilder;
+import io.kroxylicious.kubernetes.filter.api.v1alpha1.KafkaProtocolFilter;
+import io.kroxylicious.kubernetes.operator.assertj.ConditionListAssert;
 import io.kroxylicious.kubernetes.operator.assertj.KafkaServiceStatusAssert;
 
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
 
 @EnableKubernetesMockClient(crud = true)
 @ExtendWith(MockitoExtension.class)
@@ -40,8 +53,75 @@ class KafkaServiceReconcilerTest {
     KubernetesClient kubeClient;
     KubernetesMockServer mockServer;
 
-    @Mock
-    Context<KafkaService> context;
+    // @formatter:off
+    public static final KafkaService SERVICE = new KafkaServiceBuilder()
+            .withNewMetadata()
+                .withName("test-service")
+                .withGeneration(OBSERVED_GENERATION)
+            .endMetadata()
+            .withNewSpec()
+                .withNewTls()
+                    .withNewCertificateRef()
+                        .withName("my-secret")
+                    .endCertificateRef()
+                    .withNewTrustAnchorRef()
+                        .withName("my-configmap")
+                    .endTrustAnchorRef()
+                .endTls()
+            .endSpec()
+            .build();
+
+    public static final Secret TLS_SECRET = new SecretBuilder()
+            .withNewMetadata()
+                .withName("my-secret")
+                .withGeneration(42L)
+            .endMetadata()
+            .withType("kubernetes.io/tls")
+            .addToData("tls.crt", "value")
+            .addToData("tls.key", "value")
+            .build();
+
+    public static final Secret UNSUPPORTED_SECRET = new SecretBuilder()
+            .withNewMetadata()
+                .withName("my-secret")
+                .withGeneration(42L)
+            .endMetadata()
+            .addToData("key", "value")
+            .build();
+
+    public static final ConfigMap PEM_CONFIG_MAP = new ConfigMapBuilder()
+            .withNewMetadata()
+                .withName("my-configmap")
+                .withGeneration(42L)
+            .endMetadata()
+            .addToData("ca-bundle.pem", "value")
+            .build();
+
+    public static final ConfigMap P12_CONFIG_MAP = new ConfigMapBuilder()
+            .withNewMetadata()
+                .withName("my-configmap")
+                .withGeneration(42L)
+            .endMetadata()
+            .addToData("ca-bundle.p12", "value")
+            .build();
+
+    public static final ConfigMap JKS_CONFIG_MAP = new ConfigMapBuilder()
+            .withNewMetadata()
+                .withName("my-configmap")
+                .withGeneration(42L)
+            .endMetadata()
+            .addToData("ca-bundle.jks", "value")
+            .build();
+
+    public static final ConfigMap UNSUPPORTED_CONFIG_MAP = new ConfigMapBuilder()
+            .withNewMetadata()
+                .withName("my-configmap")
+                .withGeneration(42L)
+            .endMetadata()
+            .addToData("unsuppor.ted", "value")
+            .build();
+
+    // @formatter:on
 
     private KafkaServiceReconciler kafkaProtocolFilterReconciler;
 
@@ -51,7 +131,7 @@ void setUp() {
     }
 
     @Test
-    void shouldSetAcceptedToUnknown() {
+    void shouldSetResolvedRefsToUnknown() {
         // given
         final KafkaService kafkaService = new KafkaServiceBuilder().withNewMetadata().withGeneration(OBSERVED_GENERATION).endMetadata().build();
 
@@ -69,15 +149,227 @@ void shouldSetAcceptedToUnknown() {
                 .hasObservedGenerationInSyncWithMetadataOf(kafkaService)
                 .singleCondition()
                 .hasObservedGenerationInSyncWithMetadataOf(kafkaService)
-                .isAcceptedUnknown("java.lang.RuntimeException", "Boom!")
+                .isResolvedRefsUnknown("java.lang.RuntimeException", "Boom!")
                 .hasLastTransitionTime(TEST_CLOCK.instant());
 
     }
 
-    @Test
-    void shouldSetAcceptedToTrue() {
-        // Given
-        final KafkaService kafkaService = new KafkaServiceBuilder().withNewMetadata().withGeneration(OBSERVED_GENERATION).endMetadata().build();
+    static List<Arguments> shouldSetResolvedRefs() {
+        List<Arguments> result = new ArrayList<>();
+        // no client cert, no trust
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.empty());
+            result.add(Arguments.argumentSet("no tls",
+                    new KafkaServiceBuilder(SERVICE).editSpec().withTls(null).endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsTrue()));
+        }
+
+        ////////////
+        // trust bundle cases....
+
+        // no client cert, dangling trust bundle
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.empty());
+            result.add(Arguments.argumentSet("dangling trust bundle",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls().withCertificateRef(null).endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_REFS_NOT_FOUND,
+                                    "spec.tls.trustAnchorRef: referenced resource not found")));
+        }
+
+        // no client cert, unsupported trust bundle kind
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.of(UNSUPPORTED_CONFIG_MAP));
+            result.add(Arguments.argumentSet("unsupported trustAnchorRef kind",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withCertificateRef(null)
+                            .editTrustAnchorRef().withKind("Unsupported").endTrustAnchorRef()
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_REF_GROUP_KIND_NOT_SUPPORTED,
+                                    "spec.tls.trustAnchorRef supports referents: configmaps")));
+        }
+
+        // no client cert, trust bundle ref missing key
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.of(UNSUPPORTED_CONFIG_MAP));
+            result.add(Arguments.argumentSet("trust bundle ref missing key",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withCertificateRef(null)
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_INVALID,
+                                    "spec.tls.trustAnchorRef must specify 'key'")));
+        }
+
+        // no client cert, unsupported trust bundle content
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.of(UNSUPPORTED_CONFIG_MAP));
+            result.add(Arguments.argumentSet("unsupported trust bundle contents",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withCertificateRef(null)
+                            .editTrustAnchorRef().withKey("ca-bundle.bob").endTrustAnchorRef()
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_INVALID,
+                                    "spec.tls.trustAnchorRef.key should end with .pem, .p12 or .jks")));
+        }
+
+        // no client cert, pem trust bundle
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.of(PEM_CONFIG_MAP));
+            result.add(Arguments.argumentSet("pem trust bundle",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withCertificateRef(null)
+                            .editTrustAnchorRef().withKey("ca-bundle.pem").endTrustAnchorRef()
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsTrue()));
+        }
+        // no client cert, p12 trust bundle
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.of(P12_CONFIG_MAP));
+            result.add(Arguments.argumentSet("p12 trust bundle",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withCertificateRef(null)
+                            .editTrustAnchorRef().withKey("ca-bundle.p12").endTrustAnchorRef()
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsTrue()));
+        }
+        // no client cert, jks trust bundle
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.of(JKS_CONFIG_MAP));
+            result.add(Arguments.argumentSet("jks trust bundle",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withCertificateRef(null)
+                            .editTrustAnchorRef().withKey("ca-bundle.jks").endTrustAnchorRef()
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsTrue()));
+        }
+
+        ////////////
+        // client certificate cases....
+
+        // dangling client cert, no trust
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.empty());
+            result.add(Arguments.argumentSet("dangling client cert",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls().withTrustAnchorRef(null).endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_REFS_NOT_FOUND,
+                                    "spec.tls.certificateRef: referenced resource not found")));
+        }
+
+        // unsupported client cert kind, no trust
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.empty());
+            mockGetConfigMap(context, Optional.empty());
+            result.add(Arguments.argumentSet("unsupported client cert kind",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withTrustAnchorRef(null)
+                            .editCertificateRef().withKind("Unsupported").endCertificateRef()
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_REF_GROUP_KIND_NOT_SUPPORTED,
+                                    "spec.tls.certificateRef supports referents: secrets")));
+        }
+
+        // unsupported client cert in Secret, no trust
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.of(UNSUPPORTED_SECRET));
+            mockGetConfigMap(context, Optional.empty());
+            result.add(Arguments.argumentSet("unsupported client cert Secret content",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls()
+                            .withTrustAnchorRef(null)
+                            .endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsFalse(
+                                    Condition.REASON_INVALID_REFERENCED_RESOURCE,
+                                    "spec.tls.certificateRef: referenced secret should have 'type: kubernetes.io/tls'")));
+        }
+
+        // tls client cert, no trust
+        {
+            Context<KafkaProtocolFilter> context = mock(Context.class);
+            mockGetSecret(context, Optional.of(TLS_SECRET));
+            mockGetConfigMap(context, Optional.empty());
+            result.add(Arguments.argumentSet("tls client cert",
+                    new KafkaServiceBuilder(SERVICE).editSpec().editTls().withTrustAnchorRef(null).endTls().endSpec().build(),
+                    context,
+                    (Consumer<ConditionListAssert>) conditionList -> conditionList
+                            .singleElement()
+                            .isResolvedRefsTrue()));
+        }
+
+        return result;
+    }
+
+    private static void mockGetConfigMap(
+                                         Context<KafkaProtocolFilter> noClientCertAndNoTrust,
+                                         Optional<ConfigMap> empty) {
+        when(noClientCertAndNoTrust.getSecondaryResource(ConfigMap.class, KafkaServiceReconciler.CONFIG_MAPS_EVENT_SOURCE_NAME)).thenReturn(empty);
+    }
+
+    private static void mockGetSecret(
+                                      Context<KafkaProtocolFilter> noClientCertAndNoTrust,
+                                      Optional<Secret> optional) {
+        when(noClientCertAndNoTrust.getSecondaryResource(Secret.class, KafkaServiceReconciler.SECRETS_EVENT_SOURCE_NAME)).thenReturn(optional);
+    }
+
+    @ParameterizedTest
+    @MethodSource
+    void shouldSetResolvedRefs(KafkaService kafkaService, Context<KafkaService> context, Consumer<ConditionListAssert> asserter) {
 
         // When
         final UpdateControl<KafkaService> updateControl = kafkaProtocolFilterReconciler.reconcile(kafkaService, context);
@@ -85,11 +377,9 @@ void shouldSetAcceptedToTrue() {
         // Then
         assertThat(updateControl).isNotNull();
         assertThat(updateControl.getResource()).isPresent();
-        KafkaServiceStatusAssert.assertThat(updateControl.getResource().get().getStatus())
+        var c = KafkaServiceStatusAssert.assertThat(updateControl.getResource().get().getStatus())
                 .hasObservedGenerationInSyncWithMetadataOf(kafkaService)
-                .conditionList()
-                .containsOnlyTypes(Condition.Type.Accepted)
-                .singleOfType(Condition.Type.Accepted)
-                .isAcceptedTrue();
+                .conditionList();
+        asserter.accept(c);
     }
 }
diff --git a/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/assertj/ConditionAssert.java b/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/assertj/ConditionAssert.java
index f2090f35a5..5a272fd816 100644
--- a/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/assertj/ConditionAssert.java
+++ b/kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/assertj/ConditionAssert.java
@@ -50,7 +50,7 @@ public AbstractComparableAssert<?, Condition.Type> type() {
     }
 
     public ConditionAssert hasType(Condition.Type expected) {
-        type().isEqualTo(expected);
+        type().describedAs(actual.toString()).isEqualTo(expected);
         return this;
     }
 
@@ -64,7 +64,7 @@ public ConditionAssert hasNoMessage() {
     }
 
     public ConditionAssert hasMessage(String expected) {
-        message().isEqualTo(expected);
+        message().describedAs(actual.toString()).isEqualTo(expected);
         return this;
     }
 
@@ -73,7 +73,7 @@ public AbstractStringAssert<?> reason() {
     }
 
     public ConditionAssert hasReason(String expected) {
-        reason().isEqualTo(expected);
+        reason().describedAs(actual.toString()).isEqualTo(expected);
         return this;
     }
 
@@ -82,7 +82,7 @@ public ObjectAssert<Condition.Status> status() {
     }
 
     public ConditionAssert hasStatus(Condition.Status expected) {
-        status().isEqualTo(expected);
+        status().describedAs(actual.toString()).isEqualTo(expected);
         return this;
     }
 


Reviews:
user: login: tombentley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T02:23:29Z
user: login: tombentley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T02:33:31Z
user: login: k-wall
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T17:22:43Z
user: login: k-wall
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T18:31:35Z
user: login: k-wall
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-15T18:32:05Z

Review comments:
diff_hunk: @@ -44,6 +46,7 @@ public AnyLocalRefBuilder edit() {
     private String group;
 
     @Override
+    @Nullable
path: kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/api/common/AnyLocalRef.java
user: login: tombentley
body: In another PR we should push these up to the base class, since they can be null in `AnyLocalRef`.
created_at: 2025-04-15T02:14:27Z
author_association: MEMBER
diff_hunk: @@ -8,52 +8,200 @@
 
 import java.time.Clock;
 import java.util.List;
+import java.util.Optional;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import io.fabric8.kubernetes.api.model.ConfigMap;
+import io.fabric8.kubernetes.api.model.Secret;
+import io.javaoperatorsdk.operator.api.config.informer.InformerEventSourceConfiguration;
 import io.javaoperatorsdk.operator.api.reconciler.Context;
 import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusUpdateControl;
 import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
 import io.javaoperatorsdk.operator.api.reconciler.UpdateControl;
 import io.javaoperatorsdk.operator.processing.event.source.EventSource;
+import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource;
 
+import io.kroxylicious.kubernetes.api.common.AnyLocalRef;
 import io.kroxylicious.kubernetes.api.common.Condition;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaService;
+import io.kroxylicious.kubernetes.api.v1alpha1.KafkaServiceSpec;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.Tls;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.tls.TrustAnchorRef;
 
+import edu.umd.cs.findbugs.annotations.Nullable;
+
+import static io.kroxylicious.kubernetes.api.common.Condition.Type.ResolvedRefs;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.name;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.namespace;
 
+/**
+ * <p>Reconciles a {@link KafkaService} by checking whether resources referred to in {@code spec.tls.certificateRefs}
+ * actually exist, setting a {@link Condition.Type#ResolvedRefs} {@link Condition} accordingly.</p>
+ *
+ * <p>Because a service CR is not uniquely associated with a {@code KafkaProxy} it's not possible
+ * to set an {@code Accepted} condition on CR instances.</p>
+ */
 public final class KafkaServiceReconciler implements
         io.javaoperatorsdk.operator.api.reconciler.Reconciler<KafkaService> {
 
     private static final Logger LOGGER = LoggerFactory.getLogger(KafkaServiceReconciler.class);
 
+    public static final String SECRETS_EVENT_SOURCE_NAME = "secrets";
+    public static final String CONFIG_MAPS_EVENT_SOURCE_NAME = "configmaps";
+
+    public static AnyLocalRef asRef(TrustAnchorRef reference) {
path: kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java
user: login: tombentley
body: I ended up adding a `key` property to the `trustAnchorRef`, so we can't treat it directly as a `LocalRef`. This method papers over the difference.
created_at: 2025-04-15T02:15:46Z
author_association: MEMBER
diff_hunk: @@ -8,52 +8,200 @@
 
 import java.time.Clock;
 import java.util.List;
+import java.util.Optional;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import io.fabric8.kubernetes.api.model.ConfigMap;
+import io.fabric8.kubernetes.api.model.Secret;
+import io.javaoperatorsdk.operator.api.config.informer.InformerEventSourceConfiguration;
 import io.javaoperatorsdk.operator.api.reconciler.Context;
 import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusUpdateControl;
 import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
 import io.javaoperatorsdk.operator.api.reconciler.UpdateControl;
 import io.javaoperatorsdk.operator.processing.event.source.EventSource;
+import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource;
 
+import io.kroxylicious.kubernetes.api.common.AnyLocalRef;
 import io.kroxylicious.kubernetes.api.common.Condition;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaService;
+import io.kroxylicious.kubernetes.api.v1alpha1.KafkaServiceSpec;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.Tls;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.tls.TrustAnchorRef;
 
+import edu.umd.cs.findbugs.annotations.Nullable;
+
+import static io.kroxylicious.kubernetes.api.common.Condition.Type.ResolvedRefs;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.name;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.namespace;
 
+/**
+ * <p>Reconciles a {@link KafkaService} by checking whether resources referred to in {@code spec.tls.certificateRefs}
+ * actually exist, setting a {@link Condition.Type#ResolvedRefs} {@link Condition} accordingly.</p>
+ *
+ * <p>Because a service CR is not uniquely associated with a {@code KafkaProxy} it's not possible
+ * to set an {@code Accepted} condition on CR instances.</p>
+ */
 public final class KafkaServiceReconciler implements
         io.javaoperatorsdk.operator.api.reconciler.Reconciler<KafkaService> {
 
     private static final Logger LOGGER = LoggerFactory.getLogger(KafkaServiceReconciler.class);
 
+    public static final String SECRETS_EVENT_SOURCE_NAME = "secrets";
+    public static final String CONFIG_MAPS_EVENT_SOURCE_NAME = "configmaps";
+
+    public static AnyLocalRef asRef(TrustAnchorRef reference) {
+        AnyLocalRef result = new AnyLocalRef();
+        result.setGroup(reference.getGroup());
+        result.setKind(reference.getKind());
+        result.setName(reference.getName());
+        return result;
+    }
+
     private final KafkaServiceStatusFactory statusFactory;
 
     public KafkaServiceReconciler(Clock clock) {
         this.statusFactory = new KafkaServiceStatusFactory(clock);
     }
 
+    @Override
+    public List<EventSource<?, KafkaService>> prepareEventSources(EventSourceContext<KafkaService> context) {
+        InformerEventSourceConfiguration<Secret> serviceToSecret = InformerEventSourceConfiguration.from(
+                Secret.class,
+                KafkaService.class)
+                .withName(SECRETS_EVENT_SOURCE_NAME)
+                .withPrimaryToSecondaryMapper((KafkaService cluster) -> ResourcesUtil.localRefAsResourceId(cluster, cluster.getSpec().getTls().getCertificateRef()))
+                .withSecondaryToPrimaryMapper(proxy -> ResourcesUtil.findReferrers(context,
+                        proxy,
+                        KafkaService.class,
+                        service -> service.getSpec().getTls().getCertificateRef()))
+                .build();
+        InformerEventSourceConfiguration<ConfigMap> serviceToConfigMap = InformerEventSourceConfiguration.from(
+                ConfigMap.class,
+                KafkaService.class)
+                .withName(CONFIG_MAPS_EVENT_SOURCE_NAME)
+                .withPrimaryToSecondaryMapper(
+                        (KafkaService cluster) -> ResourcesUtil.localRefAsResourceId(cluster, asRef(cluster.getSpec().getTls().getTrustAnchorRef())))
+                .withSecondaryToPrimaryMapper(proxy -> ResourcesUtil.findReferrers(context,
+                        proxy,
+                        KafkaService.class,
+                        service -> asRef(service.getSpec().getTls().getTrustAnchorRef())))
+                .build();
+        return List.of(
+                new InformerEventSource<>(serviceToSecret, context),
+                new InformerEventSource<>(serviceToConfigMap, context));
+    }
+
     @Override
     public UpdateControl<KafkaService> reconcile(KafkaService service, Context<KafkaService> context) {
-        UpdateControl<KafkaService> uc = UpdateControl.patchStatus(statusFactory.newTrueConditionStatusPatch(service, Condition.Type.Accepted));
+
+        KafkaService updatedService = null;
+
+        var trustAnchorRefOpt = Optional.ofNullable(service.getSpec())
+                .map(KafkaServiceSpec::getTls)
+                .map(Tls::getTrustAnchorRef);
+        if (trustAnchorRefOpt.isPresent()) {
+            updatedService = checkTrustAnchorRef(service, context, trustAnchorRefOpt.get());
+        }
+
+        if (updatedService == null) {
+            var certRefOpt = Optional.ofNullable(service.getSpec())
+                    .map(KafkaServiceSpec::getTls)
+                    .map(Tls::getCertificateRef);
+            if (certRefOpt.isPresent()) {
+                updatedService = checkCertRef(service, context, certRefOpt.get());
+            }
+        }
+
+        if (updatedService == null) {
+            updatedService = statusFactory.newTrueConditionStatusPatch(service, ResolvedRefs);
+        }
+
+        UpdateControl<KafkaService> uc = UpdateControl.patchStatus(updatedService);
+
         if (LOGGER.isInfoEnabled()) {
             LOGGER.info("Completed reconciliation of {}/{}", namespace(service), name(service));
         }
         return uc;
     }
 
-    @Override
-    public List<EventSource<?, KafkaService>> prepareEventSources(EventSourceContext<KafkaService> context) {
-        return io.javaoperatorsdk.operator.api.reconciler.Reconciler.super.prepareEventSources(context);
+    @Nullable
+    private KafkaService checkTrustAnchorRef(KafkaService service,
+                                             Context<KafkaService> context,
+                                             TrustAnchorRef trustAnchorRef) {
+        String path = "spec.tls.trustAnchorRef";
+        if ("ConfigMap".equals(Optional.ofNullable(trustAnchorRef.getKind()).orElse("ConfigMap"))
+                && Optional.ofNullable(trustAnchorRef.getGroup()).orElse("").isEmpty()) {
+            Optional<ConfigMap> configMapOpt = context.getSecondaryResource(ConfigMap.class, CONFIG_MAPS_EVENT_SOURCE_NAME);
+            if (configMapOpt.isEmpty()) {
+                return statusFactory.newFalseConditionStatusPatch(service, ResolvedRefs,
+                        Condition.REASON_REFS_NOT_FOUND,
+                        path + ": referenced resource not found");
+            }
+            else {
+                String key = trustAnchorRef.getKey();
+                if (key == null) {
path: kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java
user: login: tombentley
body: Strictly speaking this should be enforced from the CRD schema, but it seems harmless to have a check in the code.
created_at: 2025-04-15T02:16:54Z
author_association: MEMBER
diff_hunk: @@ -44,6 +44,103 @@ spec:
                   type: string
                   minLength: 1
                   pattern: ^(([^:]+:[0-9]{1,5}),)*([^:]+:[0-9]{1,5})$
+                tls:
path: kroxylicious-operator/src/main/resources/META-INF/fabric8/kafkaservices.kroxylicious.io-v1.yml
user: login: tombentley
body: This API is rather under-specified in the [design doc](https://github.com/kroxylicious/design/blob/f5810bab03a306545fcac692ff1023a680ba776f/proposals/001-kroxylicious-operator-api-v1alpha.md). Some decisions I made here include:

* That `trustAnchorRef` is a single ref, not an array or refs
* That `trustAnchorRef` should have a required `key`, rather trying to handle `ConfigMaps` with multiple entries. This is compatible with cert-manager's trust manager, for example
* That `trustAnchorRef` should default to being a `ConfigMap`. This is compatible with cert-manager's trust manager, for example.
* That we will only support Secrets of `type: kubernetes.io/tls` in this PR, rather than trying to support arbitrary Secrets with key and cert `data` in a variety of formats. We can certainly add that in a later PR.
created_at: 2025-04-15T02:22:31Z
author_association: MEMBER
diff_hunk: @@ -44,6 +44,103 @@ spec:
                   type: string
                   minLength: 1
                   pattern: ^(([^:]+:[0-9]{1,5}),)*([^:]+:[0-9]{1,5})$
+                tls:
+                  description: Configuration for TLS connections made to the Kafka cluster
+                  type: object
+                  properties:
+                    certificateRef:
+                      description: |
+                        A reference to a resource holding the TLS client certificate to be used when 
+                        connecting to the Kafka cluster. 
+                        
+                        When the resource is of `kind: Secret` it is expected to have `type: kubernetes.io/tls` with 
+                        keys `tls.key` and `tls.crt`. See https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets.
+                      type: object
+                      required: [ "name" ]
+                      properties:
+                        kind:
+                          description: The API kind of the resource containing the TLS client certificate.
+                          type: string
+                          default: Secret
+                        group:
+                          description: The API group of the resource containing the TLS client certificate.
+                          type: string
+                          default: ""
+                        name:
+                          description: The name of the resource containing the TLS client certificate.
+                          type: string
+                    trustAnchorRef:
+                      description: |
+                        A reference to a resource holding the trusted CA certificates for validation of
+                        a Kafka broker's TLS server certificate.
+                        If not specified the proxy will use the trust store inherited from the platform.
+                      type: object
+                      required: [ "name", "key" ]
+                      properties:
+                        kind:
+                          description: The API kind of the resource containing the TLS client certificate.
+                          type: string
+                          default: ConfigMap
+                        group:
+                          description: The API group of the resource containing the TLS client certificate.
+                          type: string
+                          default: ""
+                        name:
+                          description: The name of the resource containing the TLS client certificate.
+                          type: string
+                        key:
+                          description: | 
+                            The name of the key identifying the certificate bundle within the resource. 
+                            Supported formats for the bundle are: PEM, PKCS#12 and JKS.
+                          type: string
+                    protocols:
+                      description: |
+                        The allowed or denied TLS protocols.
+                        Either `allowed` or `denied` can be given.
+                        The supported protocols are those supported by the proxy JVM.
+                      type: object
+                      minProperties: 1
+                      maxProperties: 1
path: kroxylicious-operator/src/main/resources/META-INF/fabric8/kafkaservices.kroxylicious.io-v1.yml
user: login: tombentley
body: specify either/or, but not both.
created_at: 2025-04-15T02:22:57Z
author_association: MEMBER
diff_hunk: @@ -8,52 +8,200 @@
 
 import java.time.Clock;
 import java.util.List;
+import java.util.Optional;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import io.fabric8.kubernetes.api.model.ConfigMap;
+import io.fabric8.kubernetes.api.model.Secret;
+import io.javaoperatorsdk.operator.api.config.informer.InformerEventSourceConfiguration;
 import io.javaoperatorsdk.operator.api.reconciler.Context;
 import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusUpdateControl;
 import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
 import io.javaoperatorsdk.operator.api.reconciler.UpdateControl;
 import io.javaoperatorsdk.operator.processing.event.source.EventSource;
+import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource;
 
+import io.kroxylicious.kubernetes.api.common.AnyLocalRef;
 import io.kroxylicious.kubernetes.api.common.Condition;
 import io.kroxylicious.kubernetes.api.v1alpha1.KafkaService;
+import io.kroxylicious.kubernetes.api.v1alpha1.KafkaServiceSpec;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.Tls;
+import io.kroxylicious.kubernetes.api.v1alpha1.kafkaservicespec.tls.TrustAnchorRef;
 
+import edu.umd.cs.findbugs.annotations.Nullable;
+
+import static io.kroxylicious.kubernetes.api.common.Condition.Type.ResolvedRefs;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.name;
 import static io.kroxylicious.kubernetes.operator.ResourcesUtil.namespace;
 
+/**
+ * <p>Reconciles a {@link KafkaService} by checking whether resources referred to in {@code spec.tls.certificateRefs}
+ * actually exist, setting a {@link Condition.Type#ResolvedRefs} {@link Condition} accordingly.</p>
+ *
+ * <p>Because a service CR is not uniquely associated with a {@code KafkaProxy} it's not possible
+ * to set an {@code Accepted} condition on CR instances.</p>
+ */
 public final class KafkaServiceReconciler implements
         io.javaoperatorsdk.operator.api.reconciler.Reconciler<KafkaService> {
 
     private static final Logger LOGGER = LoggerFactory.getLogger(KafkaServiceReconciler.class);
 
+    public static final String SECRETS_EVENT_SOURCE_NAME = "secrets";
+    public static final String CONFIG_MAPS_EVENT_SOURCE_NAME = "configmaps";
+
+    public static AnyLocalRef asRef(TrustAnchorRef reference) {
+        AnyLocalRef result = new AnyLocalRef();
+        result.setGroup(reference.getGroup());
+        result.setKind(reference.getKind());
+        result.setName(reference.getName());
+        return result;
+    }
+
     private final KafkaServiceStatusFactory statusFactory;
 
     public KafkaServiceReconciler(Clock clock) {
         this.statusFactory = new KafkaServiceStatusFactory(clock);
     }
 
+    @Override
+    public List<EventSource<?, KafkaService>> prepareEventSources(EventSourceContext<KafkaService> context) {
+        InformerEventSourceConfiguration<Secret> serviceToSecret = InformerEventSourceConfiguration.from(
+                Secret.class,
+                KafkaService.class)
+                .withName(SECRETS_EVENT_SOURCE_NAME)
+                .withPrimaryToSecondaryMapper((KafkaService cluster) -> ResourcesUtil.localRefAsResourceId(cluster, cluster.getSpec().getTls().getCertificateRef()))
+                .withSecondaryToPrimaryMapper(proxy -> ResourcesUtil.findReferrers(context,
+                        proxy,
+                        KafkaService.class,
+                        service -> service.getSpec().getTls().getCertificateRef()))
+                .build();
+        InformerEventSourceConfiguration<ConfigMap> serviceToConfigMap = InformerEventSourceConfiguration.from(
+                ConfigMap.class,
+                KafkaService.class)
+                .withName(CONFIG_MAPS_EVENT_SOURCE_NAME)
+                .withPrimaryToSecondaryMapper(
+                        (KafkaService cluster) -> ResourcesUtil.localRefAsResourceId(cluster, asRef(cluster.getSpec().getTls().getTrustAnchorRef())))
+                .withSecondaryToPrimaryMapper(proxy -> ResourcesUtil.findReferrers(context,
+                        proxy,
+                        KafkaService.class,
+                        service -> asRef(service.getSpec().getTls().getTrustAnchorRef())))
+                .build();
+        return List.of(
+                new InformerEventSource<>(serviceToSecret, context),
+                new InformerEventSource<>(serviceToConfigMap, context));
+    }
+
     @Override
     public UpdateControl<KafkaService> reconcile(KafkaService service, Context<KafkaService> context) {
path: kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconciler.java
user: login: tombentley
body: Note that we can't validate the protocols or cipher suites here because we don't know what's supported by the JVM of the operand. Instead we're going to have to pass them though unvalidated. It might be good if the proxy terminated in a well-defined way in this situation (ideally logging what _is_ supported), but it feels like it's a rough edge we'll have to live with.
created_at: 2025-04-15T02:33:31Z
author_association: MEMBER
diff_hunk: @@ -42,7 +42,7 @@
 
 /**
  * <p>Reconciles a {@link KafkaProtocolFilter} by checking whether the {@link Secret}s
- * and {@link ConfigMap}s refered to in interpolated expressions actually exist, setting a
+ * and {@link ConfigMap}s referred to in interpolated expressions actually exist, setting a
path: kroxylicious-operator/src/main/java/io/kroxylicious/kubernetes/operator/KafkaProtocolFilterReconciler.java
user: login: k-wall
body: the HTTP specification has a lot to answer for
created_at: 2025-04-15T17:22:43Z
author_association: MEMBER
diff_hunk: @@ -56,7 +56,7 @@ void stopOperator() {
     }
 
path: kroxylicious-operator/src/test/java/io/kroxylicious/kubernetes/operator/KafkaServiceReconcilerIT.java
user: login: k-wall
body: I'd like to see an IT that exercises the transitions when TLS bearing secret/configmap is a) removed, b) added late.
created_at: 2025-04-15T18:31:34Z
author_association: MEMBER

PR comments:
user: login: k-wall
created_at: 2025-04-15T18:37:18Z
body: > don't hook it up to proxy configuration generation yet (this will follow in a later PR).

I have a preference for delivering PRs that deliver a complete feature.  So if it were me, I'd have sliced things up the other way - completely implement say trustanchors (including the configuration of the proxy), then delivered clientcert as a separate PR.  I'm **not** suggesting you change tack on this PR.

author_association: MEMBER
user: login: sonarqubecloud[bot]
created_at: 2025-04-15T23:11:18Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=kroxylicious_kroxylicious&pullRequest=2058) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=kroxylicious_kroxylicious&pullRequest=2058&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=kroxylicious_kroxylicious&pullRequest=2058&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=kroxylicious_kroxylicious&pullRequest=2058&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [86.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=kroxylicious_kroxylicious&pullRequest=2058&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=kroxylicious_kroxylicious&pullRequest=2058&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=kroxylicious_kroxylicious&pullRequest=2058)


author_association: NONE