SE This Week: Using etcd as Kubernetes App Database
Tags: kubernetes, etcd, webapp, spring
Published: 2020-05-03 19:32:40


### About
This tutorial will show you how to build a web application using etcd as NoSQL database.

### What is [etcd](https://etcd.io/)?
etcd is a distributed reliable key-value store for the most critical data of a distributed system. etcd is written in Go. Since it is using grpc for user-facing API implementation, that explains why we choose protobuf in our web application. 
If you are familiar with Kubernetes, you should know that etcd is a well-known implementation of Kubernetes object storage. That is actually how etcd caught my attention. 

### Build an etcd web app.
Ok. Let's see how to build a web application using etcd, spring boot and protobuf.

* Use spring initializr to create a sample web project (add web dependency in).

* Add protobuf plugin and dependency, and etcd java client.
```
            <plugin>
				<groupId>org.xolstice.maven.plugins</groupId>
				<artifactId>protobuf-maven-plugin</artifactId>
				<version>0.6.1</version>
				<executions>
					<execution>
						<goals>
							<goal>compile</goal>
							<goal>test-compile</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
``` 

```
         <dependency>
			<groupId>io.etcd</groupId>
			<artifactId>jetcd-core</artifactId>
			<version>${jetcd-version}</version>
		</dependency>
		<dependency>
			<groupId>com.google.protobuf</groupId>
			<artifactId>protobuf-java</artifactId>
			<version>3.4.0</version>
		</dependency>
```

* Create a protobuf file for communication between etcd and app. It is like schema in SQL world, json in MongoDB world.
```
syntax = "proto2";
package com.example.proto.model;

option optimize_for = SPEED;

message DemoMessage {
  optional string name = 1;
  optional string body = 2;
}
```

* Let's create an etcd initialization class, which will help you create the connection between the web app and etcd.
```
@Component
public class DbInitializer {
    @Value("${etcd.url}")
    String etcdUrl;

    public KV initDB() {
        Client client = Client.builder().endpoints(etcdUrl)
                .build();
        KV kvClient = client.getKVClient();
        return kvClient;
    }
}
```

* Now, let's create a dao layer.
```
@Repository
public class MessageDaoImpl implements MessageDao{

    private KV kvClient;

    public MessageDaoImpl(@Autowired DbInitializer initializer) {
        kvClient = initializer.initDB();
    }

    @Override
    public Message.DemoMessage get(String key) throws ExecutionException, InterruptedException, InvalidProtocolBufferException {
        ByteSequence keySeq = ByteSequence.from(key.getBytes());
        CompletableFuture<GetResponse> getFuture = kvClient.get(keySeq);
        GetResponse response = getFuture.get();
        byte[] data = response.getKvs().get(0).getValue().getBytes();

        return Message.DemoMessage.parseFrom(data);
    }

    @Override
    public void delete(String key) throws ExecutionException, InterruptedException {
        ByteSequence keySeq = ByteSequence.from(key.getBytes());
        kvClient.delete(keySeq).get();

    }

    @Override
    public void add(String key, Message.DemoMessage demoMessage) throws ExecutionException, InterruptedException {
        ByteSequence keySeq = ByteSequence.from(key.getBytes());
        ByteSequence value = ByteSequence.from(demoMessage.toByteArray());
        kvClient.put(keySeq, value).get();
    }

    @Override
    public void update(String key, Message.DemoMessage demoMessage) throws ExecutionException, InterruptedException {
        ByteSequence keySeq = ByteSequence.from(key.getBytes());
        ByteSequence value = ByteSequence.from(demoMessage.toByteArray());
        kvClient.put(keySeq, value).get();
    }
}
```

* Once you have the dao layer, it shouldn't make any difference like how you create a web application. Please refer to this [GitHub](https://github.com/jiayanguo/etcdapp) repo for the completed source code. 

### How to deploy it to a Kubernetes cluster?
I will use minikube for this tutorial. It shouldn't make any difference if you choose other Kubernetes cluster, like EKS, AKS, GKS.

* Start minikube.
  `minikube start`
  If you don't have minikube yet, follow this [guide]( https://kubernetes.io/docs/tasks/tools/install-minikube/).

* Deploy etcd. 
  Download the etcd yaml file from [here](https://github.com/jiayanguo/etcdapp/blob/master/etcd.yaml). And then, run `kubectl apply -f etcd.yaml`

* Deploy our web application.
  Download the deployment yaml file from [here](https://github.com/jiayanguo/etcdapp/blob/master/deployment.yaml). And then, run `kubectl apply -f deployment.yaml`

* So far, you should be all good. But, let's expose the web application, so we can test it in our browser. Run this command. `minikube service etcd-service`

### Test
Now, we are ready to test our web application. The previous step should give you a host IP address. Please replace the IP address below with what you get. You call choose postman or any other rest API client to run the CURLs.

```
Add record
curl -X PUT \
  'http://192.168.99.102:32001/message/1?name=jguo&body=I%20am%20a%20software%20engineer' \
  -H 'cache-control: no-cache'

Get Record
curl -X GET \
  http://192.168.99.102:32001/message/1 \
  -H 'cache-control: no-cache'

Update Record
curl -X POST \
  'http://192.168.99.102:32001/message/1?name=jiayan' \
  -H 'cache-control: no-cache'

Delete Record
curl -X DELETE \
  http://192.168.99.102:32001/message/1 \
  -H 'cache-control: no-cache'
```
  
### Wrap Up
If you are using Kubernetes to run your web applications and looking for a containerized NoSQL DB, take etcd into your account. The purpose of this post is not to persuade you to choose etcd as your solution. Since choosing a DB it usually has a lot of concerns, making a smart decision. 


### Reference 
https://github.com/jiayanguo/etcdapp
https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/
https://github.com/etcd-io/etcd/tree/master/hack/kubernetes-deploy
https://github.com/etcd-io/jetcd