Creating an NFS Server with Vagrant and Archlinux for Kubernetes Cluster
Tags: kubernetes, vagrant, archlinux, sre
Published: 2020-04-06 02:29:18

---
title: Creating an NFS Server with Vagrant and Archlinux for Kubernetes Cluster
published: true
description: This post talks about how to provision an NFS server to either be used in a kubernetes cluster or mounted for any other purpose
tags: #kubernetes,#vagrant,#archlinux,#sre
---

Disclaimer: This post is originally meant to house my process for creating an NFS server (it was a huge toil). As such a lot of things might be omitted or assumed. If you find it useful and you need more insight or questions, please feel free to drop a comment and ask your question.

Also, there are several cloud providers for storage online. Almost if not all major cloud providers have the service available. However in the spirit of curiosity and how is it done, I decided to create mine.
(Also, it's a pet project and might not even see the light of the day so why waste $)

That said, let's move on.

While learning kubernetes recently from [here](https://app.pluralsight.com/library/courses/kubernetes-developers-core-concepts/table-of-contents), I began undertaking the task of migrating one of my laravel pet projects to kubernetes.

Everything was fine, until I got to the point where I needed to persist some of my essential files (logs,database,images etc).

First, I provision my k8s cluster using vagrant with the knowledge I learnt from [here](https://www.youtube.com/watch?v=wPdIBeWJJsg&list=PL34sAs7_26wODP4j6owN-36Vg-KbACgkT&index=2). Then I added an extra machine to the Vagrantfile to provision an archlinux box as well.

Here is the NFS server box:
```
#filename=Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |cf|
  # NFS Server
  cf.vm.define "nfs-server" do |nfs|
    nfs.vm.box = "archlinux/archlinux"
    nfs.vm.hostname = "nfs-server.example.com"
    nfs.vm.network "private_network", ip: "172.42.42.99"
    nfs.vm.provider "virtualbox" do |n|
      n.name = "nfs-server"
      n.memory = 1024
      n.cpus = 1
    end
    nfs.vm.provision "shell",path: "bootstrap_nfs.sh"
  end
end
```

This box defines a machine named `nfs-server` with the following properties:
* OS: `archlinux` official image from vagrant box collection
* Hostname: "nfs-server.example.com", This machine can be reached by this name within the network as well.
* IPAddress: The entire cluster runs on a private address, and since I will like this to be available to the cluster alone, I add it to the same address space.
* Provider: I use virtualbox, various options exists as defined in [official vagrant documentation](https://www.vagrantup.com/docs/vagrantfile/machine_settings.html#config-vm-provider). Various providers are listed [here](https://www.vagrantup.com/docs/providers/)
* Name: uniquely identify the box with the list of machines (I have a lot of them for my k8s cluster)
* Memory: Assigned memory to this machine. 1Gb in this case
* cpus: Just 1, it can be increased for more performance though.

`bootstrap_nfs.sh` is the script that is used to provision the machine.

```
#filename=bootstrap_nfs.sh

# Update hosts file
echo "[TASK 1] Update /etc/hosts file"
cat >>/etc/hosts<<EOF
172.42.42.99 nfs-server.example.com nfs-server
172.42.42.100 lmaster.example.com lmaster
172.42.42.101 lworker1.example.com lworker1
172.42.42.102 lworker2.example.com lworker2
EOF

echo "[TASK 2] Download and install NFS server"
yes| sudo pacman -S nfs-utils

echo "[TASK 3] Create a kubedata directory"
mkdir -p /srv/nfs/kubedata
mkdir -p /srv/nfs/kubedata/db
mkdir -p /srv/nfs/kubedata/storage
mkdir -p /srv/nfs/kubedata/logs

echo "[TASK 4] Update the shared folder access"
chmod -R 777 /srv/nfs/kubedata

echo "[TASK 5] Make the kubedata directory available on the network"
cat >>/etc/exports<<EOF
/srv/nfs/kubedata    *(rw,sync,no_subtree_check,no_root_squash)
EOF

echo "[TASK 6] Export the updates"
sudo exportfs -rav

echo "[TASK 7] Enable NFS Server"
sudo systemctl enable nfs-server

echo "[TASK 8] Start NFS Server"
sudo systemctl start nfs-server

```
_I added the Task number with a description to show what is being done and also as a documentation._

The hosts needs to be updated so that the nfs server can reach the cluster nodes. Each node will also have the corresponding entry, it's just a way of finding each other within the private network.

On to my PersistentVolume definition on the cluster,

```
#filename=storage_volume.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  namespace: app
  name: app-storage-pv
  labels:
    tier: storage
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
    - ReadOnlyMany
  storageClassName: app-storage-pv
  mountOptions:
    - nfsvers=4.1
  nfs:
    path: "/srv/nfs/kubedata/storage"
    server: nfs-server.example.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-storage-pvc
  namespace: app
  labels:
    tier: storage
    app: app-storage
spec:
  storageClassName: app-storage-pv
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
```
I created a PV that maps to the exposed folder `/srv/nfs/kubedata/storage` and using the FQDN within the network, I can use `nfs-server.example.com` for my server name (please ping this from one of your nodes to ensure it is reachable).
 

```
#filename=logs_volume.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  namespace: app
  name: app-logs-pv
  labels:
    tier: storage
    app: app-logs
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
    - ReadOnlyMany
  storageClassName: app-logs-pv
  mountOptions:
    - nfsvers=4.1
  nfs:
    path: "/srv/nfs/kubedata/logs"
    server: nfs-server.example.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-logs-pvc
  namespace: app
  labels:
    tier: storage
    app: app-logs
spec:
  storageClassName: app-logs-pv
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

```

This creates a PersistentVolume mapping to path `/srv/nfs/kubedata/logs` on our NFS server using the same `server` details `nfs-server.example.com`.

Now comes our deployment file
```
#filename=deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-backend
  namespace: app
  labels:
    app: app-backend
    tier: backend

spec:
  selector:
    matchLabels:
      app: app-backend
  template:
    metadata:
      labels:
        app: app-backend
        tier: backend

    spec:
      containers:
        - name: app-backend
          image: image_name
          env:
            - name: DB_PASSWORD
              value: "password"
          ports:
            - containerPort: 80
          livenessProbe:
            tcpSocket:
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 15
          readinessProbe:
            tcpSocket:
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 10
          volumeMounts:
            - mountPath: /app/storage/app
              name: quest-storage
            - mountPath: /app/storage/logs
              name: app-logs

      volumes:
        - name: app-storage
          persistentVolumeClaim:
            claimName: app-storage-pvc
        - name: app-logs
          persistentVolumeClaim:
            claimName: app-logs-pvc
---

apiVersion: v1
kind: Service
metadata:
  name: app-backend
  namespace: app
  labels:
    tier: backend
spec:
  selector:
    app: app-backend
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 80
```

In Laravel, the storage folder is mostly used to house file uploads and other generated files. [official docs](https://laravel.com/docs/7.x/filesystem) 

Given pods are `mortal` you don't want to lose customer's profile pictures , image uploads or any other essential file when your pod(s) go down (they do that often), we mount the nfs `/srv/kubedata/storage/` of the nfs server to Laravel `storage/app` on our pod.

Application logs are also super important, Laravel uses monolog and you don't want your log file to go down with each pod hence we also mount `/srv/nfs/kubedata/logs` to Laravel's `storage/logs/`.

Once this mount is successful, you can always be sure that even if your cluster goes down, as long as your NFS server is still up and running, your files will be safe and you can bring your cluster back to life without changing a thing.

PS: There might be a better way to mount the entire storage folder, but I didn't want to add some files that are not needed eg, the `view cache` files. This might be useful for some other people though; and it might make sense to add them to the mount.

Again, there are various cloud provider options that can be used and kubernetes supports most of them, this is just to get my pet project up and running in kubernetes and to also add another _feather to my cap_ during this lockdown.

Go and explore!!!


Thank you!