Detecting Idle Listener In Kafka
By: Debashisa Mohanty
Tags: Cosumer, Event Listener, Idle, Spring Boot, Kafka

Noticing idle consumer is an important aspect of asynchronous messaging. There are many scenarios where you may want to take some action in case of a message consumer is inactive for a period of time.

You can achieve this via Spring framework through @EventListener.

Below I will demonstrate how to achieve this for Kafka Messaging.

I have Kafka and Zookeeper running on my local docker conatiner. I am using IntelliJ Idea as my editor.

First thing first, need to create a topic in Kafka instance running on docker through command line.

NOTE: You can create the topic through java code using NewTopic class or let Spring auto create for you if you have enable auto create as true in your broker config.

2. I have created a Springboot project from start.spring.io with proper dependencies.

3. Below is how my Kafka Listener Config looks like.

@Configuration

@EnableKafka

public class KafkaListenerConfig {



private KafkaProperties kafkaProperties;



@Autowired

public KafkaListenerConfig(KafkaProperties kafkaProperties){

this.kafkaProperties=kafkaProperties;

}



@Bean

public ConsumerFactory<String,String> consumerFactory(){

return new DefaultKafkaConsumerFactory(kafkaProperties.buildConsumerProperties());

}



@Bean

public ConcurrentKafkaListenerContainerFactory<String,String> kafkaListenerContainerFactory(){

ConcurrentKafkaListenerContainerFactory<String,String> container=

new ConcurrentKafkaListenerContainerFactory<>();

container.setConsumerFactory(consumerFactory());

container.getContainerProperties().setPollTimeout(100);



/**

* Here we are setting idle interval. When the listener is idle for below mentioned idle

* time interval in milliseconds an event will

* be published.

*/

container.getContainerProperties().setIdleEventInterval(60000L);

return container;

}





}

4. Below is my Kafka Message listener and Idle Event Listener. As suggested by Spring documentation, it is a good idea to distinguish your Event Listener container via some condition.

@Slf4j

@Component

public class KafkaMessageListener {



@KafkaListener(id = "idle-listener-test", topics = "idlelistener-test", containerFactory = "kafkaListenerContainerFactory")

public void listenMessage(ConsumerRecord<String,String> consumerRecord){

log.info("Listener message received @ ::"+LocalDateTime.now() + " Message as :: "+consumerRecord.value());

}



@EventListener(condition = "event.listenerId.startsWith('idle-listener-test')")

public void idleEventHandler(ListenerContainerIdleEvent event){

log.info("Idle Event Handler received message @ :: "+ LocalDateTime.now());

}







}

5. Consumer config details are supplied through application.yml.

6. Now it’s time to build and start the Springboot application.

7. Once application is started and consumer is active, I will send some messages into application through kafka-console-producer and observe the application console.

NOTE: You can choose to produce message through any alternate way you may choose.

Application Console shows below how the Idle Event Handler receives the Idle notification and does it’s intended job.

Conclusion: This is a handy and very convenient way of detecting Idle consumers and take action on them.