Using Kubernetes Deployments to deliver SmartOS iPXE Infrastructure
By: Joseph D. Marhee
Tags: System Administration, Docker, Kubernetes, DevOps, Smartos

A couple of times before, I wrote about my various approaches to using iPXE to boot SmartOS:

I recently elected to move this (rather small) part of my operation to Kubernetes for a couple of reasons, but mostly because I wanted a fast way to refresh the served version of SmartOS archive, and I wanted it to be quickly reproducible, something my previous approach to using configuration management for this task that became sort of arduous to keep up with:

Since my approach basically comes down to 2 components, an Nginx frontend, and a backend manager for pulling down and mounting the platform archive to somewhere nginx can reach it relative to the iPXE script, Kubernetes offered a pretty straightforward way to do this.

I only needed to create a base Docker image that would contain my iPXE script, and an informational webpage:



dhcp

set base-url

kernel ${base-url}/smartos/smartos/platform/i86pc/kernel/amd64/unix -B smartos=true,console=ttyb,ttyb-mode="115200,8,n,1,-"

module ${base-url}/smartos/smartos/platform/i86pc/amd64/boot_archive type=rootfs name=ramdisk

boot #!ipxedhcpset base-url http://sdc-ipxe.east.gourmet.yoga kernel ${base-url}/smartos/smartos/platform/i86pc/kernel/amd64/unix -B smartos=true,console=ttyb,ttyb-mode="115200,8,n,1,-"module ${base-url}/smartos/smartos/platform/i86pc/amd64/boot_archive type=rootfs name=ramdiskboot

and the Dockerfile to build this image:

FROM nginx COPY index.html /usr/share/nginx/html/index.html

COPY smartos.ipxe /usr/share/nginx/html/smartos.ipxe

Build and push it to the registry.

The tricky part, and where Kubernetes could be the most help, would be where I pull down the archive, place it on a volume, and then mount it to my above Nginx container. I did this by having an InitContainer pull the archive and decompress it to a volume (I used a hostPath, but since this is not data I wish to persist between schedulings, an emptyDir would suffice as well):



kind: Deployment

metadata:

name: ipxe-deployment

labels:

app: ipxe

spec:

replicas: 2

selector:

matchLabels:

app: ipxe

template:

metadata:

labels:

app: ipxe

spec:

containers:

...

initContainers:

- name: config-data

image: ubuntu:xenial

command: ["/bin/sh","-c"]

args: ["apt update; apt install -y wget tar; wget https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/platform-latest.tgz; tar xvf platform-latest.tgz -C /data; mkdir /data/smartos; mv /data/platform* /data/smartos/platform"]

volumeMounts:

- mountPath: /data

name: ipxe-data

volumes:

- name: ipxe-data

hostPath:

path: /mnt/kube-data/ipxe/

type: DirectoryOrCreate

imagePullSecrets:

- name: regcred apiVersion: apps/v1kind: Deploymentmetadata:name: ipxe-deploymentlabels:app: ipxespec:replicas: 2selector:matchLabels:app: ipxetemplate:metadata:labels:app: ipxespec:containers:initContainers:- name: config-dataimage: ubuntu:xenialvolumeMounts:- mountPath: /dataname: ipxe-datavolumes:- name: ipxe-datahostPath:path: /mnt/kube-data/ipxe/type: DirectoryOrCreateimagePullSecrets:- name: regcred

The above has, before my Nginx container is deployed, declared that an InitContainer will run first to extract the archive to the volume, and once that has completed, will proceed to create the Nginx container:

spec:

containers:

- name: ipxe

image: coolregistryusa.biz/jmarhee/smartos-ipxe:latest

imagePullPolicy: Always

ports:

- containerPort: 80

volumeMounts:

- mountPath: /usr/share/nginx/html/smartos

name: ipxe-data

So, once the InitContainer completes, the archive volume will be mounted to /usr/share/nginx/html/smartos which is where the ipxe script expects the path to the kernel to be.

The last piece is the Kubernetes Service to expose this to the network, so, if you use a provider like Packet (see the above linked pieces for consuming this endpoint) to boot using Custom iPXE, you can boot from this endpoint:

---

kind: Service

apiVersion: v1

metadata:

name: ipxe-service

spec:

selector:

app: ipxe

ports:

- protocol: TCP

port: 8085

targetPort: 80

type: LoadBalancer

Further Reading