Spring Cloud Config, Spring Cloud Bus & Kafka | How to set up automatic updates of your configuration
Tags: springboot, springcloud, kafka
Published: 2023-12-23 10:56:55

---
title: Spring Cloud Config, Spring Cloud Bus & Kafka | How to set up automatic updates of your configuration
published: true
description: In this article, I will show you how to achieve automatic updates with Spring Cloud Bus and Kafka
tags: SpringBoot, SpringCloud, Kafka
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-12-23 10:53 +0000
---

In this article, I will show you how to achieve automatic updates with Spring Cloud Bus and Kafka

The Spring Cloud Bus, according to documentation:
>  Spring Cloud Bus links nodes of a distributed system with a lightweight message broker. This can then broadcast state changes (e.g., configuration changes) or other management instructions. AMQP and Kafka broker implementations are included in the project. Alternatively, any [Spring Cloud Stream](https://spring.io/projects/spring-cloud-stream) binder on the classpath will work out of the box as a transport.

I will start from common practice nowadays without a message broker.

First, we need to create a repository with your app’s configuration. There is no rocket science to create a new repository. After that, we can edit with web IDE, clone, add, commit, and push the app’s configuration to the remote repository.

![Example of repository](https://cdn-images-1.medium.com/max/4760/1*mPalzun7PZqMvu5PDj-0JA.png)

Example of configuration in repository

    echo:
      message:
        text: 'Hello, world!'

The second part is to create a cloud config server and set up the correct configuration for this server. We must set up a remote repository and credentials to make the configuration accessible.

![The creation of Spring Cloud Config Service Intellij IDEA](https://cdn-images-1.medium.com/max/2820/1*sWSILWoGolRwiZnE7fIgSw.png)

![We only need Spring Config Server dependency for now](https://cdn-images-1.medium.com/max/2824/1*zZfshyb0eIXvZ3hXlSgSAw.png)

Before developing a service that will be consuming configuration, we need to add an annotation that enables the config server and provides configuration.

    package io.vrnsky.cloudconfigservice;
    
    import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
    import org.springframework.cloud.config.server.EnableConfigServer;
    
    @SpringBootApplication
    @EnableConfigServer
    public class CloudConfigServiceApplication {
    
        public static void main(String[] args) {
            SpringApplication.run(CloudConfigServiceApplication.class, args);
        }
    
    }

    server:
      port: 8888
    spring:
      cloud:
        config:
          enabled: true
          server:
            git:
              uri: https://github.com/vrnsky/medium-config
              username: vrnsky
              password: your_github_personal_access_token here
              try-master-branch: false
              clone-on-start: true
              search-paths:
                - medium-service
    logging:
      level:
        org.springframework.cloud: DEBUG

After a successful start, we can check if the configuration is available.

    GitCredentialsProviderFactory : Constructing UsernamePasswordCredentialsProvider for URI https://github.com/vrnsky/medium-config

    curl "http://localhost:8888/medium-service/main"
    {"name":"medium-service","profiles":["main"],"label":null,"version":"f3c546784ab421046174a4119f2ca250184e740b","state":null,"propertySources":[{"name":"https://github.com/vrnsky/medium-config/medium-service/application.yml","source":{"echo.message.text":"Hello, world!"}}]}%

![Creation of service, which will be](https://cdn-images-1.medium.com/max/3168/1*h1RDqKY539IUOE7acG-V0Q.png)

![Dependecies of medium service](https://cdn-images-1.medium.com/max/3208/1*5l30sU-VqO5MqumyJK1tHw.png)

Now, let’s configure our service to interact with the cloud-config server. I will configure the application to use the config server application running on my machine.

    spring:
      application:
        name: medium-service
      cloud:
        config:
          enabled: true
          uri: http://localhost:8888

The next part is to create configuration properties of the medium service.

    package io.vrnsky.mediumservice.config;
    
    import lombok.Data;
    import org.springframework.boot.context.properties.ConfigurationProperties;
    import org.springframework.context.annotation.Configuration;
    
    @Data
    @Configuration
    @ConfigurationProperties(prefix = "echo.message")
    public class MediumServiceConfig {
    
        private String text;
    }

Now, let’s create a controller that will return the message value from the configuration.

    package io.vrnsky.mediumservice.controller;
    
    import io.vrnsky.mediumservice.config.MediumServiceConfig;
    import lombok.RequiredArgsConstructor;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.RestController;
    
    @RestController
    @RequiredArgsConstructor
    public class MediumController {
    
        private final MediumServiceConfig config;
    
        @GetMapping("/sayHello")
        public String getMessage() {
            return config.getText();
        }
    }

Let’s check if the property has been read from the cloud config server.

    curl http://localhost:8080/sayHello
    Hello, world!

After successfully managing to run both the service and cloud config server, we are ready to move with adding Spring Cloud Bus integration.

First, we need to bootstrap with docker-compose Kafka and Zookeeper.

    version: "3"
    services:
      zookeeper:
        image: confluentinc/cp-zookeeper:latest
        environment:
          ZOOKEEPER_CLIENT_PORT: 2181
        ports:
          - "22181:2181"
    
      kafka:
        image: confluentinc/cp-kafka:latest
        depends_on:
          - zookeeper
        environment:
          KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,EXTERNAL://localhost:29092
          KAFKA_LISTENERS: PLAINTEXT://:9092,EXTERNAL://:29092
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT
          KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
        ports:
          - "9092:9092"
          - "29092:29092"

Start Kafka and Zookeeper with the following command:

    docker compose up

The next thing we will add is to configure our Cloud Config Server and our service to push and consume events from the message broker.

In your cloud config server, pom.xml add the following dependencies:

            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-config-monitor</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-stream-kafka</artifactId>
            </dependency>

The next step is to add properties in application.yml inside the resources folder of the config server.

    spring:
      kafka:
        bootstrap-servers:
          - http://localhost:29092
      cloud:
        bus:
          enabled: true

Now, try to run the cloud config server and check if it succeeded in connecting with the message broker. You will see similar log messages if the connection has been established successfully.

    .7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137-2, groupId=anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137] Adding newly assigned partitions: springCloudBus-0
    2023-12-21T13:31:23.119+08:00  INFO 7578 --- [container-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137-2, groupId=anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137] Found no committed offset for partition springCloudBus-0
    2023-12-21T13:31:23.123+08:00  INFO 7578 --- [container-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137-2, groupId=anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137] Found no committed offset for partition springCloudBus-0
    2023-12-21T13:31:23.133+08:00  INFO 7578 --- [container-0-C-1] o.a.k.c.c.internals.SubscriptionState    : [Consumer clientId=consumer-anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137-2, groupId=anonymous.7c65c8b8-caaa-45e1-b5bb-2f9c8eb3b137] Resetting offset for partition springCloudBus-0 to position FetchPosition{offset=3, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:29092 (id: 1001 rack: null)], epoch=

Now, move on to the service side; in the pom.xml of service, add the following dependency:

     <dependency>
       <groupId>org.springframework.cloud</groupId>
       <artifactId>spring-cloud-starter-bus-kafka</artifactId>
     </dependency>

The next step is configuring the service to listen to messages from the message broker.

    spring:
      config:
        import: optional:configserver:http://localhost:8888
      application:
        name: medium-service
      cloud:
        config:
          enabled: true
          uri: http://localhost:8888
        stream:
          kafka:
            binder:
              brokers:
                - http://localhost:29092
        bus:
          trace:
            enabled: true
          refresh:
            enabled: true
          env:
            enabled: true

Now, we can start our config server and server. Please ensure that Kafka and Zookeeper have been created before bootstrapping the cloud config server and service.

The automatic updates use webhooks from version control system platforms such as GitHub and GitLab. In this article, for correct working, the services should be deployed somewhere, like Cloud Platform Providers, but to keep things simple, we will do it on a local machine. 
You can create a webhook inside the repository’s settings section and get an idea of what information can be obtained from the webhook payload. I have used [https://webhook.site](https://webhook.site) to get webhooks from the medium-config repository.

Let’s change our property in a repository and manually trigger the configuration update.

    echo:
      message:
        text: 'Hello, Medium'

You may see [https://webhook.site](https://webhook.site), which has successfully caught the webhook. Copy the payload of the webhook we are going to use for the update configuration.

It’s time to trigger the update by following the command:

     curl -X POST 'http://localhost:8888/monitor?path=medium-service" -d '{webhook_payload}'

You should see something similar in the logs of the cloud-config server. The logs tell us the message has been sent to the broker.

     o.s.c.s.m.DirectWithAttributesChannel    : postSend (sent=true) on channel 'bean 'springCloudBusInput'', message: GenericMessage [payload=byte[370], headers={kafka_offset=5, scst_nativeHeadersPresent=true, kafka_consumer=org.springframework.kafka.core

In logs of service, you should see the following logs:

    2023-12-21T13:47:55.412+08:00  INFO 4389 --- [medium-service] [container-0-C-1] o.s.c.c.c.ConfigServerConfigDataLoader   : Located environment: name=medium-service, profiles=[default], label=null, version=a4ea5778de48d08b36e6a8ee310cf2e76ebde68f, state=null
    2023-12-21T13:47:55.438+08:00  INFO 4389 --- [medium-service] [container-0-C-1] o.s.cloud.bus.event.RefreshListener      : Keys refreshed [config.client.version, echo.message.text]

To check that the configuration was updated, we can again curl the sayHello method of the medium service.

    curl http://localhost:8080/sayHello
    Hello, Medium

## References

 1. [Spring Cloud Bus Documentation](https://spring.io/projects/spring-cloud-bus/)

 2. [Spring Cloud Config Documentation](https://docs.spring.io/spring-cloud-config/docs/current/reference/html/)