Project: Apache Heron (App Definition and Development / Streaming & Messaging)
File: compiling-docker.md

id: version-0.20.5-incubating-compiling-docker title: Compiling With Docker sidebar_label: Compiling With Docker original_id: compiling-docker <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> For developing Heron, you will need to compile it for the environment that you want to use it in. If you'd like to use Docker to create that build environment, Heron provides a convenient script to make that process easier. Currently debian11 and Ubuntu 20.04 are actively being supported. There is also limited support for Ubuntu 18.04, and CentOS 8. If you need another platform there are instructions for adding new ones . If you are running Docker in a virtual machine (VM), it is recommended that you adjust your settings to help speed up the build. To do this, open and go to the container in which Docker is running (usually "default" or whatever name you used to create the VM), click on the VM, and then click on Settings. Note: You will need to stop the VM before modifying these settings. Heron provides a `build-arfifacts.sh` script for Docker located in the `docker` folder. To run that script: ```bash $ cd /path/to/heron/repo $ docker/build-artifacts.sh ``` Running the script by itself will display usage information: ``` Script to build heron docker image for different platforms Input - directory containing the artifacts from the directory <artifact-directory> Output - docker image tar file saved in the directory <artifact-directory> Usage: ./docker/scripts/build-docker.sh <platform> <version_string> <artifact-directory> [-s|--squash] Argument options: <platform>: darwin, debian11, ubuntu20.04, rocky8 <version_string>: Version of Heron build, e.g. v0.17.5.1-rc <artifact-directory>: Location of compiled Heron artifact Example: ./build-docker.sh ubuntu20.04 0.12.0 ~/ubuntu NOTE: If running on OSX, the output directory will need to be under /Users so virtualbox has access to. ``` The following arguments are required: `platform` Currently we are focused on supporting the `debian11` and `ubuntu20.04` platforms. We also support building Heron locally on OSX. You can specify this as listing `darwin` as the platform. All options are: `darwin` `rocky8` `debian11` `ubuntu18.04` `ubuntu20.04` You can add other platforms using the [instructions below](#contributing-new-environments). `version-string` The Heron release for which you'd like to build artifacts. `output-directory` The directory in which you'd like the release to be built. Here's an example usage: ```bash $ docker/scripts/build-artifacts.sh debian11 0.22.1-incubating ~/heron-release ``` This will build a Docker container specific to debian11, create a source tarball of the Heron repository, run a full release build of Heron, and then copy the artifacts into the `~/heron-release` directory. Optionally, you can also include a tarball of the Heron source if you have

By default, the script will create a tarball of the current source in the Heron repo and use that to build the artifacts. Note: If you are running on Mac OS X, Docker must be run inside a VM. Therefore, you must make sure that both the source tarball and destination directory are somewhere under your home directory. For example, you cannot output the Heron artifacts to `/tmp` because `/tmp` refers to the directory inside the VM, not on the host machine. Your home directory, however, is automatically linked in to the VM and can be accessed normally. After the build has completed, you can go to your output directory and see all of the generated artifacts: ```bash $ ls ~/heron-release heron-0.22.1-incubating-debian11.tar heron-0.22.1-incubating-debian11.tar.gz heron-core-0.22.1-incubating-debian11.tar.gz heron-install-0.22.1-incubating-debian11.sh heron-layer-0.22.1-incubating-debian11.tar heron-tools-0.22.1-incubating-debian11.tar.gz ``` In case you want to have a development environment instead of making a full build, Heron provides two helper scripts for you. It could be convenient if you don't want to set up all the libraries and tools on your machine directly. The following commands are to create a new docker image with a development environment and start the container based on it: ```bash $ cd /path/to/heron/repo $ docker/scripts/dev-env-create.sh heron-dev ``` After the commands, a new docker container is started with all the libraries and tools installed. The operation system is Ubuntu 18.04 by default. Now you can build Heron like: ```bash \# bazel build scripts/packages:binpkgs \# bazel build scripts/packages:tarpkgs ``` The current folder is mapped to the '/heron' directory in the container and any changes you make on the host machine will be reflected in the container. Note that when you exit the container and re-run the script, a new container will be started with a fresh new environment. When a development environment container is running, you can use the follow script to start a new terminal in the container. ```bash $ cd /path/to/heron/repo $ docker/scripts/dev-env-run.sh heron-dev ``` You'll notice that there are multiple in the `docker` directory of Heron's source code, one for each of the currently supported platforms. To add support for a new platform, add a new `Dockerfile` to that directory and append the name of the platform to the name of the file. If you'd like to add support for Debian 8, for example, add a file named `Dockerfile.debian11`. Once you've done that, follow the instructions in the [Docker documentation](https://docs.docker.com/engine/articles/dockerfile_best-practices/). You should make sure that your `Dockerfile` specifies at least all of the following: Here's an example: ```dockerfile FROM rockylinux:8.5 ``` Here's an example: ```dockerfile ENV TARGET_PLATFORM rocky ``` Here's an example: ```dockerfile RUN apt-get update && apt-get -y install \ automake \ build-essential \ cmake \ curl \ libssl-dev \ git \ libtool \ libunwind8 \ libunwind-setjmp0-dev \ python \ python2.7-dev \ python-software-properties \ software-properties-common \ python-setuptools \ unzip \ wget ``` Here's an example: ```dockerfile RUN \ echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \ add-apt-repository -y ppa:webupd8team/java && \ apt-get update && \ apt-get install -y openjdk-11-jdk-headless && \ rm -rf /var/lib/apt/lists/* ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64 ``` Here's an example: ```dockerfile RUN wget -O /tmp/bazel.sh https://github.com/bazelbuild/bazel/releases/download/0.26.0/bazel-0.26.0-installer-linux-x86_64.sh \ && chmod +x /tmp/bazel.sh \ && /tmp/bazel.sh ``` ```dockerfile ADD compile.sh /compile.sh ```