Building a portable face recognition application with Go and dlib
Tags: go, dlib, machinelearning, facerecognition
Published: 2024-02-01 13:43:08

_**TL;DR** We're going to build a portable [facial recognition microservice](https://github.com/vearutop/faces) for Linux using static linker in Go build with CGO dependencies._

One of the great things about Go is portability of applications built with it.

You can build a binary for different platforms and run it without installing any dependencies on the target machine. This becomes especially important if such a machine is an older low-end server with limited resources that would struggle to build an app or install needed shared libraries.

Building static apps is easy when all code is in Go, but it can become more complicated with CGO dependencies.

Davis King has made an awesome C++ [dlib](https://github.com/davisking/dlib) library, that can efficiently detect people faces in a photo. Kagami Hiiragi built a [go-face](https://github.com/Kagami/go-face) library that allows using dlib from a Go application.

The installation guide mentions a few dependencies needed for a successful build:
```
sudo apt-get install libdlib-dev libblas-dev libatlas-base-dev liblapack-dev libjpeg-turbo8-dev
```

By default, the build will depend on shared libraries and then, if you try to run the resulting binary somewhere else, it may fail due to missing file, like here:
```
./faces: error while loading shared libraries: libdlib.so.19: cannot open shared object file: No such file or directory
```

Linux has an `ldd` tool to inspect binary dependencies, most common problems when you try to run a binary built on another Linux machine are version mismatch of `GLIBC` and missing libraries.
```
ldd ./faces
./faces: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./faces)
	linux-vdso.so.1 (0x00007fff871b5000)
	libdlib.so.19 => not found
	libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007f35f50b0000)
	liblapack.so.3 => /usr/lib/x86_64-linux-gnu/liblapack.so.3 (0x00007f35f47f1000)
	libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f35f4589000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f35f436f000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f35f4150000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f35f3dc7000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f35f3a29000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f35f3811000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f35f3420000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f35f5672000)
	libgfortran.so.4 => /usr/lib/x86_64-linux-gnu/libgfortran.so.4 (0x00007f35f3041000)
	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f35f2e01000)
```

I'm hosting my personal [photo-blog](https://vearutop.p1cs.art) ([github](https://github.com/vearutop/photo-blog)) on a free VM in Oracle Cloud, and because I've set this machine up quite a while ago, it is stuck at Ubuntu 18.04 with only older libraries available by default. This was a reason I've tried to enable face recognition with a statically built app.

To separate fast-paced development with easy builds from complicated builds, I decided to implement facial recognition as a standalone microservice: [https://github.com/vearutop/faces](https://github.com/vearutop/faces).

Let's see if we can get rid of dynamic dependencies and improve portability with static build.

Dlib already has everything needed for a build in isolation, but by default it would dynamically link with installed libs if they are available. Because of that, I'll try to build in a clean docker environment.

Let's create a playground Dockerfile.

`./docker/Dockerfile`:

```Dockerfile
FROM ubuntu:22.04 as builder

RUN apt-get update
RUN apt-get install -y build-essential cmake curl
RUN curl -sLO https://go.dev/dl/go1.21.6.linux-amd64.tar.gz && tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz && rm -rf go1.21.6.linux-amd64.tar.gz
RUN mkdir /dlib && cd /dlib && curl -sLO http://dlib.net/files/dlib-19.24.tar.bz2 && tar xf dlib-19.24.tar.bz2
RUN cd /dlib/dlib-19.24 && mkdir build && cd build && cmake .. && cmake --build . --config Release && make install && rm -rf /dlib
```

```
docker build -f ./docker/Dockerfile -t builder .
```

Once the build is ready, we can get into a container with our app code mounted.

```
docker run --rm -v $PWD:/app -w /app -it builder /bin/bash
```
```
root@f08033dcccbc:/app# ls
LICENSE  Makefile  README.md  bin  dev_test.go  docker  faces.go  go.mod  go.sum  models  unit.coverprofile  vendor

```

I've downloaded all dependencies with `go mod vendor` to simplify operations in the container.

In order to build statically, we need to set `CGO_LDFLAGS="-static"` for the `go build`.

```
root@f08033dcccbc:/app# CGO_LDFLAGS="-static" /usr/local/go/bin/go build .
# github.com/Kagami/go-face
jpeg_mem_loader.cc:3:10: fatal error: jpeglib.h: No such file or directory
    3 | #include <jpeglib.h>
      |          ^~~~~~~~~~~
compilation terminated.
```

Build failed on a missing header file that was supposed to be installed with one of the dependencies. Let's see if we have that file somewhere in container.

```
root@f08033dcccbc:/app# find / -name '*jpeglib.h'
/usr/local/include/dlib/external/libjpeg/jpeglib.h
```

Header files are looked up in `/usr/include/` by default. For a quick and dirty fix, we can copy the missing file(s) there.

```
root@f08033dcccbc:/app# cp /usr/local/include/dlib/external/libjpeg/*.h /usr/include/
```

Let's run the build again!

```
root@f08033dcccbc:/app# CGO_LDFLAGS="-static" /usr/local/go/bin/go build .
# github.com/vearutop/faces
/usr/local/go/pkg/tool/linux_amd64/link: running g++ failed: exit status 1
/usr/bin/ld: cannot find -lblas: No such file or directory
/usr/bin/ld: cannot find -lcblas: No such file or directory
/usr/bin/ld: cannot find -llapack: No such file or directory
/usr/bin/ld: cannot find -ljpeg: No such file or directory
collect2: error: ld returned 1 exit status
```

Bad luck, it failed with another error now. Linker complains that it cannot link against a few missing libs, but all of them should already be included in `dlib` build. 

If we search our codebase (including `vendor`), we'll find this line in `face.go`:

```
// #cgo LDFLAGS: -ldlib -lblas -lcblas -llapack -ljpeg
```

This is a linker instruction with list of libs that's causing the problem now. Fortunately, with vendored deps it is super easy to change code of dependencies, so let's change that line to this:

```
// #cgo LDFLAGS: -ldlib
```

Let's run the build one more time.

```
root@f08033dcccbc:/app# CGO_LDFLAGS="-static" /usr/local/go/bin/go build .
# github.com/vearutop/faces
/usr/bin/ld: /tmp/go-link-3191459515/000010.o: in function `_cgo_9c8efe9babca_C2func_getaddrinfo':
/tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
root@f08033dcccbc:/app# ./faces -help
Usage of ./faces:
  -listen string
        listen address (default "localhost:8011")
root@f08033dcccbc:/app# 
```

It complained about something, but worked!
Let's check the dependencies now.

```
root@f08033dcccbc:/app# ldd ./faces
        not a dynamic executable
```

This looks like a nice statically built binary! 😌

Let's check if it actually works. In the same container I can run the app in background with:

```
root@f08033dcccbc:/app# ./faces &
[1] 1739
root@f08033dcccbc:/app# 
```

And then invoke request with `curl`:

```
root@f08033dcccbc:/app# curl -X 'POST' \
'http://localhost:8011/image' \
-H 'accept: application/json' \
-H 'Content-Type: multipart/form-data' \
-F 'image=@person.jpg;type=image/jpeg'
```

```json
{"elapsedSec":0.258315,"found":1,"faces":[{"Rectangle":{"Min":{"X":352,"Y":185},"Max":{"X":567,"Y":400}},"Descriptor":[-0.16648848,-0.05050624,0.106586605,-0.0867105,-0.09123391,-0.097584575,-0.046739854,-0.103373915,0.074457884,-0.105268024,0.20660394,-0.13579035,-0.2745444,0.0005242062,-0.03232275,0.18681777,-0.13113116,-0.17754263,-0.052810747,-0.05957584,0.0062686643,-0.03621107,0.011501403,0.1487859,-0.06366991,-0.33826596,-0.06331841,-0.08673793,-0.010200936,-0.0629237,0.027267495,0.11619936,-0.2607339,-0.04982499,0.01518264,0.12889145,0.02307811,-0.118307345,0.1285096,-0.048686076,-0.24529652,-0.12607978,0.136835,0.26203102,0.162219,0.034145266,0.018228233,-0.0061597005,0.040899806,-0.295318,0.0031301053,0.06259319,0.0745079,0.049838964,0.00964687,-0.27123472,0.07631222,0.060989577,-0.12530015,0.03486493,0.035399184,-0.04188027,0.04090107,-0.051638283,0.36773872,0.10492739,-0.14495152,-0.087634355,0.21060707,-0.16210485,-0.00697436,0.04431132,-0.16566163,-0.12653385,-0.31701985,-0.06338993,0.31295794,0.03408507,-0.17158867,0.076981254,-0.09508267,0.073756054,0.02041351,0.14637248,-0.0001675617,0.10626993,-0.08162568,-0.01661037,0.23682739,-0.021808863,-0.006492801,0.22029987,-0.01065092,-0.044090617,0.09562777,0.039906204,-0.05015147,-0.061895538,-0.21429531,0.028714905,-0.07911338,-0.017555084,-0.02431442,0.106665134,-0.20538758,0.08050651,0.017503517,-0.0074621206,-0.057238452,0.036879964,-0.08754097,-0.09878489,0.111212455,-0.24645737,0.15643074,0.21560076,0.10718059,0.13916788,0.05442419,0.053753562,0.024602186,-0.011599961,-0.13366313,-0.02042818,0.062051836,-0.0836075,-0.010100439,0.07831607],"Shapes":[{"X":529,"Y":253},{"X":492,"Y":251},{"X":399,"Y":237},{"X":436,"Y":245},{"X":457,"Y":309}]}]}
```

To wrap up, let's add our findings in app codebase.

If we now try to build the app outside a container with dynamic linking, the build will fail because we've removed linker instructions in vendored code. To make it work for both cases we can guard behavior with build flags. Let's remove the `// #cgo LDFLAGS: ...` line from `face.go` and create two new files instead.

`face_static.go`:

```go
//go:build static
package face

// #cgo LDFLAGS: -ldlib
import "C"
```

`face_dynamic.go`:
```go
//go:build !static
package face

// #cgo LDFLAGS: -ldlib -lblas -lcblas -llapack -ljpeg
import "C"
```

Now we'll need to add a build tag in order to build statically, but the default build will use dynamic linking.

```
root@f08033dcccbc:/app# CGO_LDFLAGS="-static" /usr/local/go/bin/go build -tags static .
```

Let's update our `Dockerfile` with more instructions to perform the actual application build.

```Dockerfile
FROM ubuntu:22.04 as builder

RUN apt-get update
RUN apt-get install -y build-essential cmake curl
RUN curl -sLO https://go.dev/dl/go1.21.6.linux-amd64.tar.gz && tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz && rm -rf go1.21.6.linux-amd64.tar.gz
RUN mkdir /dlib && cd /dlib && curl -sLO http://dlib.net/files/dlib-19.24.tar.bz2 && tar xf dlib-19.24.tar.bz2
RUN cd /dlib/dlib-19.24 && mkdir build && cd build && cmake .. && cmake --build . --config Release && make install && rm -rf /dlib

# Missing header file.
RUN cp /usr/local/include/dlib/external/libjpeg/*.h /usr/include/

# Building app.
WORKDIR /app
ADD . .
RUN CGO_LDFLAGS="-static" /usr/local/go/bin/go build -tags static .

# Exporting minimal docker image with pre-built binary.
FROM alpine
WORKDIR /root
CMD ["/bin/faces", "-listen", "0.0.0.0:80"]
COPY --from=builder /app/faces /bin/faces
```

After that we can run a clean build.
```
docker build -f ./docker/Dockerfile -t faces .
```

Now, if you need resulting binary to deploy it somewhere, you can copy it from docker image.

```
docker run -v $PWD:/opt/mount --rm faces cp /bin/faces /opt/mount/faces
```

As a result you'll have `./faces` in your current directory.


Or you can start the service on http://localhost:8000 with docker.
```
docker run --rm -p 8000:80 faces
```