A few tips for Perl on Docker and Kubernetes
Tags: perl, docker, kubernetes, devops
Published: 2020-11-29 11:49:11

@davorg started a thread on the FB [Perl Community](https://web.facebook.com/groups/perlcommunity/permalink/877301896410771/) and [Perl Programmers](https://web.facebook.com/groups/perlprogrammers/permalink/3786053688094058/) about [docker-perl](https://hub.docker.com/_/perl), and as its maintainer, I'm overdue sharing some tips about it so might as well write them here now:

### Use a custom name/tag for Perl images you use

As documented in the Docker Hub, one can use the `perl` image with something like

```sh
docker pull perl
docker run -it perl perl -E 'say "hi there from Docker!"'
```

While doing this is enough for simple or exploratory cases, it is inadequate for development or deployment scenarios, as `perl` alone will pull the latest Perl image _build_ of the latest supported Perl version (which is `5.32.0` _now_ but could be different later on.)  Fortunately, docker-perl provides _tags_ to indicate specific versions (as well as options like `:threaded`) or size variants (like `:slim`) so one can also do

```sh
docker pull perl:5.32-buster
docker run -it perl:5.32-buster perl -E 'say qq{hi again!}'
```

There is another concern though: while Docker images have tags, these tags are _floating_ in the sense that they don't always point to the same underlying image layer at creation time, as tags can be updated to point to another layer.  This is best seen in the `:latest` tag which Docker uses by default when calling images with their bare names, but this behavior is also present in any other tags like `:5` which is pointing currently to `:5.32.0`, but might later be updated to point to `:5.32.1` or even `:5.34.0`, when the [perl manifest on docker-library/official-images](https://github.com/docker-library/official-images/blob/master/library/perl) is updated.

These tags also get updated indirectly by way of updates on their _base_ images; as docker-perl uses the [`buildpack-deps`](https://hub.docker.com/_/buildpack-deps) or [`debian:slim`](https://hub.docker.com/_/debian/) base images, when these get updated for security patches, these patches will eventually make their way into the `perl` images as well through the [official-images build tooling](https://doi-janky.infosiftr.net/job/multiarch/).

For cases that require finer control of images, it might be helpful then to use a custom image name, tag, or both, when using docker-perl:

```sh
docker pull perl:5.32-slim-threaded-buster
docker tag perl:5.32-slim-threaded-buster myorg/perl:5.32
docker run -it myorg/perl:5.32 perl -e 'printf "hello from myorg/perl v%vd!\n", $^V'
```

This allows for some independence for updates and development between the `perl` published on Docker Hub and `myorg`'s custom `perl` image:

```Dockerfile
# docker build -t myorg/myapp:dev .
FROM myorg/perl:5.32
ADD . /app
WORKDIR /app
RUN cpanm --installdeps .
CMD ["perl", "myapp.pl"]
...
```

### Consider keeping a local/private registry to host Perl images

Using the perl images (especially like in the previous section) means that these will be copied into the local Docker host image storage, usually in `/var/lib/docker/images` but could vary depending on what [storage driver](https://docs.docker.com/storage/storagedriver/select-storage-driver/) your host is using.  While this is usually enough in simple cases, consider that the base `perl` image is usually big (around 700-800MB unpacked,) so pulling this image afresh over multiple container hosts on a network will probably be wasteful.

Hence it is usually recommended to have some kind of local or private _registry_ in the host or on a network if frequently working with containers (and more so if internet connectivity is slower/expensive.)  There are several approaches to do this, and I've written about these before in my old blog:

- [Composing a Docker Hub mirror and private registry](https://zakame.net/blog/2017/12/composing-a-docker-hub-mirror-and-private-registry.html)
- [Mirroring the Docker Hub](https://zakame.net/blog/2017/07/mirroring-the-docker-hub.html)

Nowadays though, there are more options for using local/private registries: for example, if you have a Kubernetes cluster running in your network (or even as a small setup like [microk8s](https://microk8s.io), or an internal Docker container cluster using [KinD](https://kind.sigs.k8s.io) or [k3d](https://k3d.io),) there's usually an addon 
that enables a local registry on the cluster.  Here's [an example](https://microk8s.io/docs/registry-built-in) for microk8s, which I extended for my own local network to be served under a [TLS ingress](https://kubernetes.github.io/ingress-nginx/examples/tls-termination/):

```sh
microk8s enable registry
microk8s kubectl apply -f - <<REGISTRY_SVC
apiVersion: v1
kind: Service
metadata:
  name: registry
  namespace: default
spec:
  # microk8s deploys registry in its own namespace, so reach out
  externalName: registry.container-registry.svc.cluster.local
  ports:
  - name: registry
    port: 5000
    protocol: TCP
    targetPort: 5000
  type: ExternalName
REGISTRY_SVC
microk8s kubectl apply -f - <<REGISTRY_INGRESS
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: registry
  namespace: default
spec:
  rules:
  # I don't own home.example btw, just for illustration;
  # I use globally-accessible subdomains pointing to private IP
  - host: registry.home.example
    http:
      paths:
      - backend:
          serviceName: registry
          servicePort: 5000
  tls:
  - hosts:
    - registry.home.example
    secretName: wildcard-home-example
REGISTRY_INGRESS
```

I have a [LetsEncrypt](https://letsencrypt.org) wildcard SSL cert installed and maintained on my microk8s via [cert-manager](https://cert-manager.io), so this lets me push to this registry anywhere on my local network as

```sh
docker push myorg/perl:5.32 registry.home.example/myorg/perl:5.32
docker push myorg/myapp:dev registry.home.example/myorg/myapp:dev
```

and on somewhere else which might be running [Podman](https://podman.io) instead:

```sh
podman run -it registry.home.example/myorg/perl:5.32 perl -V
podman run -d registry.home.example/myorg/myapp:dev
```

It also works on authoring `Dockerfile`s, especially useful when implementing GitOps workflows:

```Dockerfile
# docker build -t myorg/mojo:8.65
# docker push myorg/mojo:8.65 registry.home.example/myorg/mojo:8.65
FROM registry.home.example/myorg/perl:5.32
RUN cpanm Mojolicious@8.65
EXPOSE 3000
CMD ["/usr/local/bin/mojo", "daemon"]
```

For more sophisticated/production deployments, consider using [Harbor](https://goharbor.io/) or vendors such as [Amazon ECR](https://aws.amazon.com/ecr/), [Google Container Registry](https://cloud.google.com/container-registry/) or [Red Hat Quay](https://quay.io/).

### Use container entrypoint for proper signals handling in Perl

This is already documented in the [official-images docs](https://github.com/docker-library/docs/tree/master/perl#signal-handling-behavior-notice) for perl, but bears repeating here: containers don't provide a _parent init process_ by default, which means that when starting a new container, any [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint) or [CMD](https://docs.docker.com/engine/reference/builder/#cmd) set in the image the container will boot from will usually become the parent process within this container.  This is important to note here for Perl (and also for other languages) as these tend to fall back to their parent init process for handling signals, so without this, it can appear to become unresponsive:

```
docker run perl:5.32 perl -E 'sleep 300'
^C
[refuses to die, even if sent SIGINT like above]
```

This is particularly important for Kubernetes, as its [container pod lifecycle](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/) can use signals to probe for liveness or readiness of containers, and a running pod/container can seem to "hang" around indefinitely if these signals aren't handled correctly.

For perl, one needs to install a [`%SIG` handler](https://perldoc.pl/perlipc#Signals) if they want/need to run perl directly as PID 1:

```
docker run perl:5.32 perl -E '$SIG{TERM} = sub { $sig++; say "recv TERM" }; sleep 300; say "waking up" if $sig'
^C
waking up
recv TERM
```

For most normal deployments however, it is easier to use a tool like [tini](https://github.com/krallin/tini), [catatonit](https://github.com/openSUSE/catatonit), or [dumb-init](https://github.com/Yelp/dumb-init) for the container's `ENTRYPOINT`, alongside your perl script/executable in `CMD`, for example this small [Mojolicious](https://mojolicious.org) demo:

```Dockerfile
# docker build -t mojo-with-tini:dev .
FROM perl:5.32-buster
RUN apt-get update && \
    apt-get install -y --no-install-recommends tini && \
    cpanm Mojolicious
ENTRYPOINT ["/usr/bin/tini", "--"]
EXPOSE 3000
CMD ["/usr/local/bin/mojo", "daemon"]
```

For Kubernetes, one can also set the [`command` and `args`](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#notes) for a pod:

```sh
kubectl apply -f - <<MOJO_POD
apiVersion: v1
kind: Pod
metadata:
  name: mojo-with-tini-demo
  labels:
    purpose: demonstrate-perl-signal-handling
spec:
  containers:
  - name: mojo-with-tini-demo-container
    image: mojo-with-tini:dev
    # redundant as we already set these in the Dockerfile above,
    # but showing here for illustration
    command: ["/usr/bin/tini", "--"]
    args: ["/usr/local/bin/mojo", "daemon"]
MOJO_POD
```

### Outro

There's a few more tips on the queue that I'd write more here but this post is getting rather long now, so maybe later.

I'd love to hear more about how people use docker-perl, and especially more about how make developing Perl projects on container environments easier!