Repository: provectus/kafka-ui
Issue: unable to see brokers details in kafka-ui
Labels: type/bug
status/accepted
status/confirmed

Hi Team,

We have installed 3 zookeeper and 3 kafka node on same instance with different port and producing and consuming message using internal dns name.

Below are the details
2181 - zookeeper1.example
2182 - zookeeper2.example
2183 - zookeeper3.example

9093 - kafka1.example
9094 - kafka2.example
9095 - kafka3.example

and we are trying to use kafka-ui for this setup using docker.
Below is the configuration that we are trying.

 "KAFKA_CLUSTERS_0_NAME", "value" : "kafka-dev",
"KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS", "value" : "kafka1.example:9093,kafka2.example:9094,kafka3.example:9095",
 "KAFKA_CLUSTERS_0_ZOOKEEPER", "value" : "zookeeper1.example:2181,zookeeper2.example:2182, zookeeper3.example:2183"


But we are unable to see this brokers details in kafka-ui. Can you please help us with the configuration.


Issue comments:
Hi, thanks for reaching out. Have you tried looking at [this example](https://github.com/provectus/kafka-ui/blob/master/docker/kafka-ui.yaml)? It seems like the difference between this example and your configuration is that `bootstrapservers` value contains just one kafka address in our case. Could you try that please?
Also, what's the status of zookeeper on your kafka-ui instance? Is it on or off? Green or red?
The status of zookeeper is off  (red)
we are getting below error while clicking in broker

{"code":5000,"message":"The broker does not support DESCRIBE_CONFIGS","timestamp":1633509870299,"requestId":"712f046f","fieldsErrors":null}
> how can we add multiple address in bootstrapservers ?

you did it the right way (I guess).



> The status of zookeeper is off (red)

Could you share application (kafka-ui) logs? There seems to be an issue with that.
> we are getting below error while clicking in broker
> 
> {"code":5000,"message":"The broker does not support DESCRIBE_CONFIGS","timestamp":1633509870299,"requestId":"712f046f","fieldsErrors":null}

Sorry missed this comment. We'll take a look into this.
It seems like your broker version might be very outdated. Probably the issue is that required API is still not implemented in the version you have present.
Is that possible to upgrade it? Don't you use azure by any chance?

Also, could you please provide application logs with that message? I need the stacktraces.
Thanks for your support.

Upgrading broker to latest version resolved my issue.
how can we secure this UI with username and password ?
> Thanks for your support.
> 
> Upgrading broker to latest version resolved my issue.

That's great, glad to hear!


> how can we secure this UI with username and password ?

There are two options you have there:
1) simple username/password auth -- [reference](https://github.com/provectus/kafka-ui/blob/master/docker/kafka-ui-auth-context.yaml#L19)
2) OIDC -- [reference](https://github.com/provectus/kafka-ui/blob/master/guides/SSO.md)
Feel free to raise a new issue (or better a new discussion) if you'll have any problems with setting up authentication.
<img width="1440" alt="Screenshot 2021-10-12 at 12 37 11 PM" src="https://user-images.githubusercontent.com/39396208/136908314-e8259dac-0ee4-4268-973a-94a3ef6ec969.png">
Hi,

I am getting this error and kafka-ui unable to connect to broker
@hasan13579 could you please create a new issue for this? And please describe which auth you use and share the config.

Pull request review comments:

Patch that resolved the issue:
diff --git a/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/BrokerServiceImpl.java b/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/BrokerServiceImpl.java
index ba5d8932a07..236ea36d28f 100644
--- a/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/BrokerServiceImpl.java
+++ b/kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/BrokerServiceImpl.java
@@ -7,8 +7,8 @@
 import com.provectus.kafka.ui.model.InternalBrokerConfig;
 import com.provectus.kafka.ui.model.KafkaCluster;
 import com.provectus.kafka.ui.util.ClusterUtil;
-import java.util.ArrayList;
 import java.util.Collections;
+import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.stream.Collectors;
@@ -18,6 +18,7 @@
 import org.apache.kafka.clients.admin.DescribeConfigsOptions;
 import org.apache.kafka.common.Node;
 import org.apache.kafka.common.config.ConfigResource;
+import org.apache.kafka.common.errors.UnsupportedVersionException;
 import org.springframework.stereotype.Service;
 import reactor.core.publisher.Flux;
 import reactor.core.publisher.Mono;
@@ -39,11 +40,15 @@ private Mono<Map<Integer, List<ConfigEntry>>> loadBrokersConfig(
         .map(ExtendedAdminClient::getAdminClient)
         .flatMap(adminClient ->
             ClusterUtil.toMono(adminClient.describeConfigs(resources,
-                new DescribeConfigsOptions().includeSynonyms(true)).all())
-                .map(config -> config.entrySet().stream()
+                    new DescribeConfigsOptions().includeSynonyms(true)).all())
+                .map(config -> config.entrySet()
+                    .stream()
                     .collect(Collectors.toMap(
                         c -> Integer.valueOf(c.getKey().name()),
-                        c -> new ArrayList<>(c.getValue().entries())))));
+                        c -> List.copyOf(c.getValue().entries())
+                    ))
+                ))
+        .onErrorResume(UnsupportedVersionException.class, (e) -> Mono.just(new HashMap<>()));
   }
 
   private Mono<List<ConfigEntry>> loadBrokersConfig(


Test patch: