Repository: provectus/kafka-ui
Issue: ACL enabled check results in an error
Labels: type/bug
scope/backend
status/accepted
area/acl

### Issue submitter TODO list

- [X] I've looked up my issue in [FAQ](https://docs.kafka-ui.provectus.io/faq/common-problems)
- [X] I've searched for an already existing issues [here](https://github.com/provectus/kafka-ui/issues)
- [X] I've tried running `master`-labeled docker image and the issue still persists there
- [X] I'm running a supported version of the application which is listed [here](https://github.com/provectus/kafka-ui/blob/master/SECURITY.md)

### Describe the bug (actual behavior)

After upgrading from version 0.6.2 to 0.7.0 I see in the KafkaUI log-file strange repeated massages (that have not been before):

org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
2023-05-15 11:14:48,017 DEBUG [parallel-3] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: KAFKATS_TEST
2023-05-15 11:15:17,964 DEBUG [parallel-4] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: KAFKATS_TEST
2023-05-15 11:15:17,971 WARN  [parallel-2] c.p.k.u.s.ReactiveAdminClient: Error checking if security enabled
org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
2023-05-15 11:15:17,992 DEBUG [parallel-3] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: KAFKATS_TEST
2023-05-15 11:15:47,964 DEBUG [parallel-4] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: KAFKATS_TEST
2023-05-15 11:15:47,972 WARN  [parallel-2] c.p.k.u.s.ReactiveAdminClient: Error checking if security enabled
org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
2023-05-15 11:15:47,990 DEBUG [parallel-3] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: KAFKATS_TEST
2023-05-15 11:16:17,964 DEBUG [parallel-4] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: KAFKATS_TEST
2023-05-15 11:16:17,972 WARN  [parallel-2] c.p.k.u.s.ReactiveAdminClient: Error checking if security enabled
org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
2023-05-15 11:16:17,992 DEBUG [parallel-3] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: KAFKATS_TEST


Kafka 'server.log' contains a synchronous repeated messages:
[2023-05-15 11:15:47,971] ERROR (getting) acls is not supported by Ranger for Kafka (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
[2023-05-15 11:15:47,971] ERROR [KafkaApi-1] Unexpected error handling request RequestHeader(apiKey=DESCRIBE_ACLS, apiVersion=2, clientId=kafka-ui-admin-1684138400-1, correlationId=49) -- DescribeAclsRequestData(resourceTypeFilter=1, resourceNameFilter=null, patternTypeFilter=1, principalFilter=null, hostFilter=null, operation=1, permissionType=1) with context RequestContext(header=RequestHeader(apiKey=DESCRIBE_ACLS, apiVersion=2, clientId=kafka-ui-admin-1684138400-1, correlationId=49), connectionId='10.120.152.240:9093-10.120.152.240:43102-474', clientAddress=/10.120.152.240, principal=User:kafkats, listenerName=ListenerName(SASL_SSL), securityProtocol=SASL_SSL, clientInformation=ClientInformation(softwareName=apache-kafka-java, softwareVersion=3.3.1), fromPrivilegedListener=true, principalSerde=Optional[org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder@6818625]) (kafka.server.KafkaApis)
java.lang.UnsupportedOperationException: (getting) acls is not supported by Ranger for Kafka
        at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.acls(RangerKafkaAuthorizer.java:334)
        at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.acls(RangerKafkaAuthorizer.java:185)
        at kafka.server.AclApis.handleDescribeAcls(AclApis.scala:70)
        at kafka.server.KafkaApis.handleDescribeAcls(KafkaApis.scala:2574)
        at kafka.server.KafkaApis.handle(KafkaApis.scala:198)
        at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:75)
        at java.lang.Thread.run(Thread.java:750)


### Expected behavior

the messages "Error checking if security enabled" are absent in the KafakUI log and a synchronous messages 'acls is not supported by Ranger for Kafka' are absent in the Kafka 'server.log'

### Your installation details

KafkaUI - [fdd9ad9](https://github.com/provectus/kafka-ui/commit/fdd9ad9) 11.05.2023, 17:02:15
KafkaUI startup script with configuration parameters:
export DYNAMIC_CONFIG_ENABLED='false'
export SECURITY_BASIC_ENABLED='false'

export SERVER_PORT='8080'
export SERVER_SSL_ENABLED='true'
export SERVER_SSL_KEY_STORE_TYPE='JKS'
export SERVER_SSL_KEY_STORE='/disk01/kafka-ui-api-v0.7.0/keystore.jks'
export SERVER_SSL_KEY_STORE_PASSWORD='***'

export AUTH_TYPE='LDAP'
export SPRING_LDAP_URLS='ldaps://****.**.*.com:3269 ldaps://****.**.*.com:3269'
export SPRING_LDAP_BASE='DC=**,DC=*,DC=com'
export SPRING_LDAP_USER_FILTER_SEARCH_BASE='DC=**,DC=*,DC=com'
export SPRING_LDAP_USER_FILTER_SEARCH_FILTER='(&(sAMAccountName={0})(|(memberOf=CN=kafka-admin,OU=Service,DC=**,DC=*,DC=com)(memberOf=CN=admin,OU=Service,DC=**,DC=*,DC=com)))'
export SPRING_LDAP_ADMIN_USER='CN=ldap-user,OU=Service,DC=**,DC=*,DC=com'
export SPRING_LDAP_ADMIN_PASSWORD='***'

export KAFKA_CLUSTERS_0_METRICS_PORT='9094'
export KAFKA_CLUSTERS_0_METRICS_SSL='false'
export KAFKA_CLUSTERS_0_METRICS_TYPE='false'

export KAFKA_CLUSTERS_0_NAME='KAFKATS_TEST'
export KAFKA_CLUSTERS_0_READONLY='false'
export KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS='kafka-tst.**.*.com:9093'
export KAFKA_CLUSTERS_0_ZOOKEEPER='kafka-tst.**.*.com.com:2182'

export KAFKA_CLUSTERS_0_SSL_TRUSTSTORELOCATION='/disk01/kafka-ui-api-v0.7.0/keystore.jks'
export KAFKA_CLUSTERS_0_SSL_TRUSTSTOREPASSWORD='***'

export KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL='SASL_SSL'
export KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM='GSSAPI'
export KAFKA_CLUSTERS_0_PROPERTIES_SASL_KERBEROS_SERVICE_NAME='kafkats'
export KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG='com.sun.security.auth.module.Krb5LoginModule required serviceName="kafkats" useKeyTab=true storeKey=true keyTab="/disk01/kafka-ui-api-v0.7.0/kafka-tst.keytab" principal="kafkats/kafka-tst.**.*.com@**.*.COM";'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_TYPE='JKS'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_LOCATION='/disk01/kafka-ui-api-v0.7.0/keystore.jks'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_PASSWORD='***'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEY_PASSWORD='***'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_TYPE='JKS'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_LOCATION='/disk01/kafka-ui-api-v0.7.0/keystore.jks'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_PASSWORD='***'
export KAFKA_CLUSTERS_0_PROPERTIES_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=''

export KAFKA_CLUSTERS_0_SCHEMAREGISTRY='https://kafka-tst.**.*.com:8081'
export KAFKA_CLUSTERS_0_SCHEMAREGISTRYSSL_KEYSTORELOCATION='/disk01/kafka-ui-api-v0.7.0/keystore.jks'
export KAFKA_CLUSTERS_0_SCHEMAREGISTRYSSL_KEYSTOREPASSWORD='***'
export KAFKA_CLUSTERS_0_SCHEMAREGISTRYSSL_KEYPASSWORD='***'
export KAFKA_CLUSTERS_0_SCHEMAREGISTRYSSL_TRUSTSTORELOCATION='/disk01/kafka-ui-api-v0.7.0/keystore.jks'
export KAFKA_CLUSTERS_0_SCHEMAREGISTRYSSL_TRUSTSTOREPASSWORD='***'

export JAVA_OPTS=" -Dzookeeper.client.secure=false -Dzookeeper.ssl.client.enable=false -Djavax.net.ssl.keyStoreType=jks -Djavax.net.ssl.keyStore=/disk01/kafka-ui-api-v0.7.0/keystore.jks -Djavax.net.ssl.keyStorePassword=*** -Djavax.net.ssl.trustStoreType=jks -Djavax.net.ssl.trustStore=/disk01/kafka-ui-api-v0.7.0/keystore.jks -Djavax.net.ssl.trustStorePassword==***"
export JAVA_OPTS="$JAVA_OPTS -Xms2g -Xmx4g -Djava.awt.headless=true --add-opens java.rmi/javax.rmi.ssl=ALL-UNNAMED"

cd /disk01/kafka-ui-api-v0.7.0
nohup /opt/java/jdk-17.0.3.1/bin/java $JAVA_OPTS -jar /disk01/kafka-ui-api-v0.7.0/kafka-ui-api-v0.7.0.jar>/disk01/kafka-ui-api-v0.7.0/kafkaui-console.log 2>&1 &

=====
Kafka - confluent-community-7.1.0 with 'ranger-kafka-plugin' enabled
broker.id=1
listeners=SASL_SSL://kafka-tst.**.*.com:9093
advertised.listeners=SASL_SSL://kafka-tst.**.*.com:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600

log.dirs=/disk01/kafkats/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1

log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000

zookeeper.connect=kafka-tst.**.*.com:2182
zookeeper.connection.timeout.ms=18000

group.initial.rebalance.delay.ms=0

sasl.kerberos.service.name=kafkats
zookeeper.ssl.client.enable=true
zookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty
zookeeper.ssl.truststore.location=/disk01/kafkats/etc/truststore.jks
zookeeper.ssl.truststore.password=***
inter.broker.listener.name=SASL_SSL
ssl.client.auth=required
ssl.keystore.location=/disk01/kafkats/etc/keystore.jks
ssl.keystore.password=***

authorizer.class.name=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer

ssl.protokol=TLS
auto.create.topics.enable=false
auto.leader.rebalance.enable=true
delete.topic.enable=true
kerberos.auth.enable=true
leader.imbalance.check.interval.seconds=300
leader.imbalance.per.broker.percentage=10
message.max.bytes=5242880
num.replica.fetchers=14
replica.fetch.max.bytes=5242880
replica.lag.time.max.ms=10000
zookeeper.session.timeout.ms=60000

sasl.mechanism.inter.broker.protokol=GSSAPI
sasl.enabled.mechanisms=GSSAPI
sasl.mechanism=GSSAPI


### Steps to reproduce

As far as I understand, the problem is that we do not use the built-in Kafka ACL mechanism for restricting access to Kafka-topics, but we use the Rager-Kafka-Plugin.
Therefore, we need the ability in KafkaUI to disable the functionality for Kafka ACL discovery from the Kafka server.

### Screenshots

_No response_

### Logs

_No response_

### Additional context

_No response_

Issue comments:
Hello there sm-shevchenko! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀
@sm-shevchenko thank you for noticing that. We will think how it can be fixed. For now you can just ignore this message, since it should not affect kui functioning. 

Pull request review comments:

Patch that resolved the issue:
diff --git a/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/ReactiveAdminClient.java b/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/ReactiveAdminClient.java
index 0b6f16a2235..9de908efa7f 100644
--- a/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/ReactiveAdminClient.java
+++ b/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/ReactiveAdminClient.java
@@ -15,6 +15,8 @@
 import com.provectus.kafka.ui.util.KafkaVersion;
 import com.provectus.kafka.ui.util.annotation.KafkaClientInternalsDependant;
 import java.io.Closeable;
+import java.time.Duration;
+import java.time.temporal.ChronoUnit;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.HashMap;
@@ -129,38 +131,41 @@ private record ConfigRelatedInfo(String version,
                                    Set<SupportedFeature> features,
                                    boolean topicDeletionIsAllowed) {
 
-    private static Mono<ConfigRelatedInfo> extract(AdminClient ac, int controllerId) {
-      return loadBrokersConfig(ac, List.of(controllerId))
-          .map(map -> map.isEmpty() ? List.<ConfigEntry>of() : map.get(controllerId))
-          .flatMap(configs -> {
-            String version = "1.0-UNKNOWN";
-            boolean topicDeletionEnabled = true;
-            for (ConfigEntry entry : configs) {
-              if (entry.name().contains("inter.broker.protocol.version")) {
-                version = entry.value();
-              }
-              if (entry.name().equals("delete.topic.enable")) {
-                topicDeletionEnabled = Boolean.parseBoolean(entry.value());
-              }
-            }
-            var builder = ConfigRelatedInfo.builder()
-                .version(version)
-                .topicDeletionIsAllowed(topicDeletionEnabled);
-            return SupportedFeature.forVersion(ac, version)
-                .map(features -> builder.features(features).build());
-          });
+    static final Duration UPDATE_DURATION = Duration.of(1, ChronoUnit.HOURS);
+
+    private static Mono<ConfigRelatedInfo> extract(AdminClient ac) {
+      return ReactiveAdminClient.describeClusterImpl(ac, Set.of())
+          .flatMap(desc -> {
+            // choosing node from which we will get configs (starting with controller)
+            var targetNodeId = Optional.ofNullable(desc.controller)
+                .map(Node::id)
+                .orElse(desc.getNodes().iterator().next().id());
+            return loadBrokersConfig(ac, List.of(targetNodeId))
+                .map(map -> map.isEmpty() ? List.<ConfigEntry>of() : map.get(targetNodeId))
+                .flatMap(configs -> {
+                  String version = "1.0-UNKNOWN";
+                  boolean topicDeletionEnabled = true;
+                  for (ConfigEntry entry : configs) {
+                    if (entry.name().contains("inter.broker.protocol.version")) {
+                      version = entry.value();
+                    }
+                    if (entry.name().equals("delete.topic.enable")) {
+                      topicDeletionEnabled = Boolean.parseBoolean(entry.value());
+                    }
+                  }
+                  final String finalVersion = version;
+                  final boolean finalTopicDeletionEnabled = topicDeletionEnabled;
+                  return SupportedFeature.forVersion(ac, version)
+                      .map(features -> new ConfigRelatedInfo(finalVersion, features, finalTopicDeletionEnabled));
+                });
+          })
+          .cache(UPDATE_DURATION);
     }
   }
 
   public static Mono<ReactiveAdminClient> create(AdminClient adminClient) {
-    return describeClusterImpl(adminClient, Set.of())
-        // choosing node from which we will get configs (starting with controller)
-        .flatMap(descr -> descr.controller != null
-            ? Mono.just(descr.controller)
-            : Mono.justOrEmpty(descr.nodes.stream().findFirst())
-        )
-        .flatMap(node -> ConfigRelatedInfo.extract(adminClient, node.id()))
-        .map(info -> new ReactiveAdminClient(adminClient, info));
+    Mono<ConfigRelatedInfo> configRelatedInfoMono = ConfigRelatedInfo.extract(adminClient);
+    return configRelatedInfoMono.map(info -> new ReactiveAdminClient(adminClient, configRelatedInfoMono, info));
   }
 
 
@@ -170,7 +175,7 @@ private static Mono<Boolean> isAuthorizedSecurityEnabled(AdminClient ac, @Nullab
         .doOnError(th -> !(th instanceof SecurityDisabledException)
                 && !(th instanceof InvalidRequestException)
                 && !(th instanceof UnsupportedVersionException),
-            th -> log.warn("Error checking if security enabled", th))
+            th -> log.debug("Error checking if security enabled", th))
         .onErrorReturn(false);
   }
 
@@ -202,6 +207,8 @@ public static <T> Mono<T> toMono(KafkaFuture<T> future) {
 
   @Getter(AccessLevel.PACKAGE) // visible for testing
   private final AdminClient client;
+  private final Mono<ConfigRelatedInfo> configRelatedInfoMono;
+
   private volatile ConfigRelatedInfo configRelatedInfo;
 
   public Set<SupportedFeature> getClusterFeatures() {
@@ -228,7 +235,7 @@ public Mono<Void> updateInternalStats(@Nullable Node controller) {
     if (controller == null) {
       return Mono.empty();
     }
-    return ConfigRelatedInfo.extract(client, controller.id())
+    return configRelatedInfoMono
         .doOnNext(info -> this.configRelatedInfo = info)
         .then();
   }


Test patch: