12 Things You Might Not Know About Buildpacks
Tags: cloudnative, docker, buildpacks
Published: 2022-12-01 13:53:16

For most buildpack users, the only thing you need to know is that running `pack build` will turn your source code into a Docker image. But the power of buildpacks goes far beyond this initial experience. Buildpacks also provide powerful features that no other image building tool can match. Let's take a look at a few of them.

## 1. Use metadata from a previous build

A buildpack can analyze information about layers it created during a previous build. For buildpack authors, this can aid in optimizing build performance by skipping steps if a buildpack determines something hasn't changed.

For example, it's common to store a checksum of a dependency in the metadata of a build layer, and then compare it to the latest version in the next build. If they match, the buildpack may skip installing that dependency and instead reuse the exact same layer from the previous build. This avoids the costly overhead of downloading large layers and moving them around unnecessarily. 

Effectively, this means a buildpack can construct a new image from both existing layers and new layers in any order.

![Layer construction](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rycmfna1q57znpjg0hxd.png)

For more information on buildpack layer caching, see the documentation on [buildpack layer types](https://buildpacks.io/docs/buildpack-author-guide/caching-strategies/#layers).

## 2. Update the operating system without re-building

Buildpacks allow you to rapidly update an image when its base image (i.e operating system) has changed. This process is called rebasing, and it avoids the need to fully rebuild the app when a new operating system or base image becomes available.

Consider an image `my-app:latest` that was originally built using the latest version of a base image, `pack/run:latest`. Running the following will update those layers of `my-app:latest` with the latest version of `pack/run:latest` without rebuilding the app layers above them:

```
pack rebase my-app:latest
```

Avoiding rebuilds is a nice performance boost for a single image, but for a fleet of hundreds (or even millions) of images, it becomes a critical operational and security tool.

## 3. Write Buildpacks in any language

Not everyone needs to write a buildpack. Most people just use the Paketo, Heroku, or Google buildpacks. But for those who need to create one from scratch, they're able to choose any language they want.

The buildpack API defines its entry-points only as executables. That means the executable can be a Python script, a Bash script, or a binary compiled from Golang code.

![Language logos](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/97cut5dkjw9utbcsrr9g.png)

The Paketo Buildpacks are mostly written in Go, while the Heroku buildpacks are written in Bash and [Rust](https://github.com/heroku/buildpacks-nodejs/blob/main/buildpacks/nodejs-engine/src/main.rs). The [libcnb](https://github.com/buildpacks/libcnb) library is a Go language binding of the Cloud Native Buildpacks API. It's supported by the project, but there are also libraries outside of the project like [libcnb.bash](https://github.com/jkutner/libcnb.bash).

## 4. Add buildpacks inline with your source code

You can create a buildpack in your source code repo very similarly to how you might drop in a `Dockerfile`. To do so, create a `project.toml` file and define your buildpack commands inline.

For example, if you wanted run a Bash script as part of your build, your `project.toml` would include something like this:

```toml
[[build.buildpacks]]
id = "me/setup-tasks"

  [build.buildpacks.script]
  api = "0.8"
  inline = "bash setup.sh"
```

For more information, see the [inline buildpacks documentation](https://buildpacks.io/docs/app-developer-guide/using-inline-buildpacks/).

## 5. Create images from scratch

Buildpacks can build images that have no underlying base layers, just as if you were using `FROM scratch`. Most of the off-the-shelf buildpacks from Paketo and Heroku don't do this because they rely on the operating system they run on, but when creating your own buildpacks it is possible with a custom base image. The `Dockerfile` for such a base image can be as simple as this:

```
FROM scratch
ENV CNB_USER_ID=1000
ENV CNB_GROUP_ID=1000
LABEL io.buildpacks.stack.id="my-scratch-img"
USER ${CNB_USER_ID}:${CNB_GROUP_ID}
```

For a more robust but still slim image, try the [Paketo "tiny" base image](https://github.com/paketo-buildpacks/jammy-tiny-stack). For more information on using custom base images see the documentation on [stacks](https://buildpacks.io/docs/concepts/components/stack/).

## 6. Use the Pack CLI as a library

The Pack CLI (i.e. `pack`) is a great tool for using buildpacks. But many end users want to bake these capabilities directly into some other tooling. Fortunately, `pack` is also a Golang library. You can include it in other Go based projects by simply running:

```
go get github.com/buildpacks/pack
```

Don't worry, this is a completely supported pattern. In fact, many of the organizations contributing to the project, including Salesforce, do this very thing.

## 7. Compose buildpacks with other buildpacks

Most advanced buildpacks aren't actually a single buildpack, but instead a composite of several buildpacks that each do smaller jobs. For example, the [Heroku Node.js buildpack](https://github.com/heroku/buildpacks-nodejs/blob/main/meta-buildpacks/nodejs/buildpack.toml) is actually a collection of several NPM and Node engine buildpacks. Here's an excerpt from its `buildpack.toml` descriptor:

```toml
[[order.group]]
id = "heroku/nodejs-engine"
version = "0.8.12"

[[order.group]]
id = "heroku/nodejs-npm"
version = "0.5.2"

[[order.group]]
id = "heroku/procfile"
version = "2.0.0"
optional = true
```

This composability makes it much easier for buildpack authors to reuse buildpacks, and allows end users to leverage them in unexpected ways.

## 8. Create a standardized SBOM

A Software-Bill-of-Materials (SBOM) lists all the software components included in an image. Buildpacks support SBOMs in [CycloneDX](https://cyclonedx.org/), [Syft](https://github.com/anchore/syft) and [SPDX](https://spdx.dev/) formats.

Each buildpack can populate the SBOM with the information about the dependencies they have provided. And end users can view that metadata with the Pack CLI by running:

```
pack sbom download your-image-name
```

For more information see the [buildpacks documentation on SBOM](https://buildpacks.io/docs/features/bill-of-materials/).

## 9. Use buildpacks without Docker installed

Pack uses the Docker API to run containerized builds and create runnable OCI images, which means it works equally well with a [secure remote daemon](https://docs.docker.com/engine/security/protect-access/) as it does with a local daemon. Using a remote daemon looks something like this:

```sh
export DOCKER_HOST=tcp://daemon-hostname:2376
export DOCKER_CERT_PATH=~/.docker/
export DOCKER_TLS_VERIFY=1

pack build my-app --path ./my-source
```

For more information on how to do this, see our past blog post [using Pack with a remote Docker daemon](https://medium.com/buildpacks/pack-with-a-remote-docker-daemon-41aab804b839).

## 10. Create reproducible builds

Given a set of inputs (source code, dependencies, etc) a buildpack can produce an image with the exact same digest (i.e. the same checksum of everything it contains) as an image from a previous build.

Reproducible builds aren't something that happens by default; each buildpack needs to be smart enough to compare the layers it creates to the metadata from previous builds. But the buildpacks API makes it possible for discerning users.

With something like Dockerfile, reproducible builds are effectively impossible because even a change to the timestamp of a file will give you a new image digest. This is why buildpacks produce images that appear to be created [40 years ago](https://medium.com/buildpacks/time-travel-with-pack-e0efd8bf05db) (you can use the `--creation-time` flag if you don't want this feature).

## 11. Run buildpacks with Github Actions

The Buildpacks project maintains a set of [github-actions](https://github.com/buildpacks/github-actions) that can be used for various buildpack related tasks. Among these is the ability to set up a job that is ready with the pack CLI. It's as simple as using this action:

```yaml
uses: buildpacks/github-actions/setup-pack@v4.1.0
```

From here, you can run pack to build images, create builders, inspect SBOM metadata, and more.

## 12. Use buildpacks with a Dockerfile

Ok, I'm cheating here because this might not be ready when I publish this blog post. Very soon, you'll be able to provide a `Dockerfile` in your buildpack build in order to customize the build process.

For more information, see the finalized RFC on [supporting Dockerfiles](https://github.com/buildpacks/rfcs/blob/main/text/0105-dockerfiles.md).