HarperDB Containerization Journey
Tags: docker, showdev, kubernetes, database
Published: 2020-11-19 16:00:34


### How I Single-Handedly Containered HarperDB

HarperDB is a simple database that is adequately configured, works as a distributed database verging towards serverless, and is significant for micro-service-oriented architecture. It enables developers to think responsibly about the data they are collecting. Companies can begin to isolate the important data from the noise, and store only the information they need where they need it. This is a possibility now more than ever before, and containers are a significant contributor to the paradigm.

Explaining containers is out of scope for this blog; the long and the short, a container is an isolated set of applications prerequisites that use the host systems resources.   
 

### Docker-ization

At HarperDB, I began ushering things to be organized in a container friendly manner early on. [HarperDB on Docker hub](https://hub.docker.com/r/harperdb/hdb) was one of our first release channels. Docker created the ability for HarperDB to spawn quickly, persist data on the host, or load data into the container for an ephemeral instance. The HarperDB application and the data store are not tightly coupled; the HarperDB application can point to any HarperDB datastore. Below is a quick Docker example of two HarperDB application instances pointing to the same database.

Create one container and make port 9925 available on the docker host instance:

```docker run -d -v /tmp/docker1/:/opt/harperdb/hdb -p 9925:9925 harperdb/hdb:latest ```

Login to the local HarperDB management Studio at `http://localhost:9925` with username: `HDB_ADMIN` and password: `password`.

Create a schema, table and add a few records.

![After Login Create Schema and Table with hash attribute (primary key) id](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605132718808_docker_create_schema_table.png) <figcaption>After Login Create Schema and Table with hash attribute (primary key) id</figcaption>

![Created two records with the + button; Rosemerry and Billie](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605132870807_docker_2dogs.png)<figcaption>Created two records with the + button; Rosemerry and Billie</figcaption>


Now create a second container attached to the same data directory

`docker run -d -v /tmp/docker1/:/opt/harperdb/hdb -p 9926:9925 harperdb/hdb:latest`

Notice the host port 9926 because 9925 is in use.

Login to a new instance and create a new record! This time at `http://localhost:9926`.


![Created Harper on localhost:9926](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605133149903_docker_add_newdog.png)<figcaption>Created Harper on localhost:9926</figcaption>


Refresh the instance on 9925!


![WhOooT!](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605133193775_docker_3_dogs.png)<figcaption>WhOooT!</figcaption>



Fun!

Is this useful in real life? Probably somewhere somehow, this can do something neat!

The example illustrates that HarperDB application containers are isolated instances that can mount any HarperDB storage.

Imagine a containers' host syncing data to a remote source like AWS S3, adding S3 data sync to update an S3 bucket periodically. Another container host instance across the planet pulls down the S3 data and starts a HarperDB instance. That is for another blog.

In real life, Docker is incredible for developers and database administrators. One-off instances are useful, but clusters and application stacks are more relevant for application tiers, and where Docker compose is helpful. However, Docker compose is not as robust as Kubernetes.

It has been my most recent project to get HarperDB on Helm, the Kubernetes package manager. For updates when that happens, follow HarperDB by joining our [Slack Channel](https://harperdbcommunity.slack.com/join/shared_invite/zt-e8w6u1pu-2UFAXl_f4ZHo7F7DVkHIDA#/) and/or [subscribing to our company updates.](https://harperdb.io/contact/)

Please go check out [HarperDB on the Docker hub](https://hub.docker.com/r/harperdb/hdb); it provides examples to get a lot more out of the Docker image and all the configurations available so far.


### Kubernetes In A Digital Ocean 

Kubernetes is a robust infrastructure that helps orchestrate running containers. Kubernetes allows users to create large-scale deployments of single or multiple containers, the life cycles of containers, and the resources containers rely on, i.e., storage, network configuration, CPU, memory. With the help of a great cloud platform, [DigitalOcean](https://www.digitalocean.com/), you can already deploy [HarperDB in their 1-click marketplace](https://marketplace.digitalocean.com/apps/harperdb). They also provide [1-click Kubernetes apps; HarperDB](https://marketplace.digitalocean.com/category/kubernetes) will be available soon; this is a preview of HarperDB deployed on DigitalOcean with [Helm.](https://helm.sh/)

This example assumes that you have following preconfigured: a [DigitalOcean account](https://cloud.digitalocean.com/registrations/new), [kubectl,](https://kubernetes.io/docs/setup/) [doctl,](https://github.com/digitalocean/doctl) and [helm](https://helm.sh/docs/intro/install/) installed in your development environment. 

Create a Kubernetes Cluster for simplicity; a two-node cluster is sufficient.

![Create two node Kubernetes cluster](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605166629782_create_cluster.png)<figcaption>Create two node Kubernetes cluster</figcaption>


Use [doctl to configure kubectl](https://www.digitalocean.com/docs/kubernetes/how-to/connect-to-cluster/) context to point to DigitalOcean Kubernetes cluster.

![Configure kubectl context for cluster](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605167278281_doctl_cluster_config.png)<figcaption>Configure kubectl context for cluster</figcaption>


For security reasons, Kubernetes providers implement Role-based access control (RBAC). It is a method of regulating access to a computer or network resources based on individual users' roles within your organization. A dependency of Helm is tiller; the following provides tiller access to resources.


    kubectl -n kube-system create serviceaccount tiller
    kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
    helm init --service-account=tiller

![Configure tiller access control for HELM](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605167691507_helm-tiller.png)<figcaption>Configure tiller access control for HELM</figcaption>


Helm charts use a YAML configuration file for resources exposed by the Kubernetes cluster as well as application specific resources. A few import configurations for HarperDB in the values.yaml.

    image:
      repository: harperdb/hdb
      tag: latest
      pullPolicy: IfNotPresent
    service:
      type: ClusterIP
      port: 9925
    volumes:
      storageClassProvisioner: dobs.csi.digitalocean.com
      storageClassName: do-block-storage
      persistentClaim: harperdb-pvc
      storage: 5Gi
      volumeName: harperdb-ps
    harperdb:
      username: HDB_ADMIN
      password: password
      cluster_enabled: true
      cluster_username: clustering
      cluster_password: password
      cluster_port: 1111
      node_name: hdb-cluster-00

Most options in the "image" and "harperdb" blocks above should be self-explanatory. The volumes block will require an explanation. Each Kubernetes provider will expose specific storage parameters. Provisioner and Class values are available in the Kubernetes dashboard in the DigitalOcean dashboard.

![Storage Provisioner and Class Values for value.yaml](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605168407855_storage_prov_class.png)<figcaption>Storage Provisioner and Class Values for value.yaml</figcaption>


The Helm chart can now create an instance of HarperDB that persists information on the host and exposes it as a service on port 9925. 

![helm install](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605168784187_Helm_install.png)<figcaption>helm install</figcaption>


The NOTES at the bottom will allow your local development environment to connect to the HarperDB instance management Studio.

![Expose studio through http://localhost:8080](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605169101014_expose_studio.png)<figcaption>Expose studio through http://localhost:8080</figcaption>



Connect to the cluster through `http://``localhost:8080`, which routes traffic to the Kubernetes HarperDB instance on port 9925.


![Create Schema, Table and Two records](https://paper-attachments.dropbox.com/s_08DD60EEAC114D5A54BC05780549766DC45D9639B77CD64E0C3587C6ABCD22E0_1605169201712_k8-harperdb.png)<figcaption>Create Schema, Table and Two records</figcaption>


In the real world, HarperDB would not generally be exposed to the world, and deployments would run over HTTPS. Application containers or Pods in Kubernetes would access HarperDB within the Kubernetes network. Again, for another blog, the HarperDB instance could be ephemeral, and the container could get a HarperDB storage copy from a remote source, then sync to the remote source as data is updated. Also, the HarperDB cluster could publish new data to another instance of HarperDB.

###Q.E.D, What was to be shown

Containers are robust, HarperDB is powerful; their powers combined provide opportunities to think about data storage and data flow in innovative ways. Docker makes it easy to containerize an application. Kubernetes provides a command and control center to build the distributed infrastructure and orchestrate container resources and life cycle. Helm gives software providers better access to Kubernetes with easy to install deployments. HarperDB is working hard to expand its offerings to other deployment channels. If you did not know, HarperDB also offers [database as a service](https://bit.ly/2KdtsDq). Your support is appreciated.