Native build procedure of Tensorflow v2.0.0 C API for RaspberryPi / arm64 devices (armhf / aarch64)

**Tensorflow-bin**　[![GitHub stars](https://img.shields.io/github/stars/PINTO0309/Tensorflow-bin.svg?style=social&label=Stars)](https://github.com/PINTO0309/Tensorflow-bin)
<a href="https://github.com/PINTO0309/Tensorflow-bin"><img src="https://github-link-card.s3.ap-northeast-1.amazonaws.com/PINTO0309/Tensorflow-bin.png" width="460px"></a>
**Bazel_bin**　[![GitHub stars](https://img.shields.io/github/stars/PINTO0309/Bazel_bin.svg?style=social&label=Stars)](https://github.com/PINTO0309/Bazel_bin)
<a href="https://github.com/PINTO0309/Bazel_bin"><img src="https://github-link-card.s3.ap-northeast-1.amazonaws.com/PINTO0309/Bazel_bin.png" width="460px"></a>

# １．Introduction
This section describes how to build Tensorflow v2.0.0 C API (RaspberryPi / arm64 devices). Since we get an error when we build according to the **[official tutorial](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/lib_package/README.md)**, I devised my own procedure. In particular, when using armhf (armv7l), an error **`undefined symbol: __atomic_fetch_add_8`** will occur, so an option must be added to the build script. My procedure can avoid **`undefined symbol: __atomic_fetch_add_8`** errors and complete the build successfully. When using RaspberryPi4, it will be completed in about 6 hours. Prebuilt binaries and installers can be downloaded from my **[Github repository](https://github.com/PINTO0309/Tensorflow-bin.git)**.

# ２．Environment

- RaspberryPi3 / 4 (Raspbian Buster / Debian Buster, armhf, glibc2.28)
- **[Scaleway arm64(aarch64) Debian Buster, 32 core / RAM 32GB](https://www.scaleway.com/en/)**
- microSD card 32GB
- Tensorflow v2.0.0
- **[Bazel 0.26.1](https://github.com/PINTO0309/Bazel_bin.git)**

# ３．Procedure

```console:Install_required_packages
$ sudo apt-get install -y \
make cmake wget curl libhdf5-dev libc-ares-dev libeigen3-dev libatomic1 \
openmpi-bin libopenmpi-dev libatlas-base-dev zip unzip
```

```console:Clone_Tensorflow
$ cd ~

$ git clone -b v2.0.0 https://github.com/tensorflow/tensorflow.git

or

$ git clone -b v1.15.0 https://github.com/tensorflow/tensorflow.git

$ cd tensorflow/tensorflow/tools/lib_package
```

```console:【Pattern_1】_Build_Tensorflow_v2.0.0_Pi3/Pi4_armhf
### Install openjdk-8-jdk armhf
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1LQUSal55R6fmawZS9zZuk6-5ZFOdUqRK" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1LQUSal55R6fmawZS9zZuk6-5ZFOdUqRK" -o adoptopenjdk-8-hotspot_8u222-b10-2_armhf.deb
$ sudo apt-get install -y ./adoptopenjdk-8-hotspot_8u222-b10-2_armhf.deb

### Install Bazel 0.26.1 armhf
$ wget https://github.com/PINTO0309/Bazel_bin/raw/master/0.26.1/Raspbian_Debian_Buster_armhf/openjdk-8-jdk/install.sh
$ ./install.sh

### Build Tensorflow
$ sudo bazel --host_jvm_args=-Xmx512m build \
--config=noaws \
--config=nohdfs \
--config=noignite \
--config=nokafka \
--config=nonccl \
--config=v2 \
--local_resources=4096.0,3.0,1.0 \   #<===[If Pi3] --local_resources=1024.0,0.5,0.5 \
--copt=-mfpu=neon-vfpv4 \
--copt=-ftree-vectorize \
--copt=-funsafe-math-optimizations \
--copt=-ftree-loop-vectorize \
--copt=-fomit-frame-pointer \
--copt=-DRASPBERRY_PI \
--host_copt=-DRASPBERRY_PI \
--linkopt=-Wl,-latomic \
--host_linkopt=-Wl,-latomic \
//tensorflow/tools/lib_package:libtensorflow
```

```console:【Pattern_2】_Build_Tensorflow_v1.15.0_Pi3/Pi4_armhf
### Install openjdk-8-jdk armhf
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1LQUSal55R6fmawZS9zZuk6-5ZFOdUqRK" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1LQUSal55R6fmawZS9zZuk6-5ZFOdUqRK" -o adoptopenjdk-8-hotspot_8u222-b10-2_armhf.deb
$ sudo apt-get install -y ./adoptopenjdk-8-hotspot_8u222-b10-2_armhf.deb

### Install Bazel 0.26.1 armhf
$ wget https://github.com/PINTO0309/Bazel_bin/raw/master/0.26.1/Raspbian_Debian_Buster_armhf/openjdk-8-jdk/install.sh
$ ./install.sh

### Build Tensorflow
$ sudo bazel --host_jvm_args=-Xmx512m build \
--config=noaws \
--config=nogcp \
--config=nohdfs \
--config=noignite \
--config=nokafka \
--config=nonccl \
--local_resources=4096.0,3.0,1.0 \   #<===[If Pi3] --local_resources=1024.0,0.5,0.5 \
--copt=-mfpu=neon-vfpv4 \
--copt=-ftree-vectorize \
--copt=-funsafe-math-optimizations \
--copt=-ftree-loop-vectorize \
--copt=-fomit-frame-pointer \
--copt=-DRASPBERRY_PI \
--host_copt=-DRASPBERRY_PI \
--linkopt=-Wl,-latomic \
--host_linkopt=-Wl,-latomic \
//tensorflow/tools/lib_package:libtensorflow
```

```console:【Pattern_3】_Build_Tensorflow_v2.0.0_Debian_Buster_aarch64
### Install openjdk-8-jdk aarch64
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1VwLxzT3EOTbhSzwvRF2H4ChTQyTQBt3x" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1VwLxzT3EOTbhSzwvRF2H4ChTQyTQBt3x" -o adoptopenjdk-8-hotspot_8u222-b10-2_arm64.deb
$ sudo apt-get install -y ./adoptopenjdk-8-hotspot_8u222-b10-2_arm64.deb

### Install Bazel 0.26.1 aarch64
$ wget https://github.com/PINTO0309/Bazel_bin/raw/master/0.26.1/Raspbian_Debian_Buster_aarch64/openjdk-8-jdk/install.sh
$ ./install.sh

### Build Tensorflow
$ sudo bazel --host_jvm_args=-Xmx512m build \
--config=noaws \
--config=nohdfs \
--config=noignite \
--config=nokafka \
--config=nonccl \
--config=v2 \
//tensorflow/tools/lib_package:libtensorflow
```

```console:【Pattern_4】_Build_Tensorflow_v1.15.0_Debian_Buster_aarch64
### Install openjdk-8-jdk aarch64
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1VwLxzT3EOTbhSzwvRF2H4ChTQyTQBt3x" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1VwLxzT3EOTbhSzwvRF2H4ChTQyTQBt3x" -o adoptopenjdk-8-hotspot_8u222-b10-2_arm64.deb
$ sudo apt-get install -y ./adoptopenjdk-8-hotspot_8u222-b10-2_arm64.deb

### Install Bazel 0.26.1 aarch64
$ wget https://github.com/PINTO0309/Bazel_bin/raw/master/0.26.1/Raspbian_Debian_Buster_aarch64/openjdk-8-jdk/install.sh
$ ./install.sh

### Build Tensorflow
$ sudo bazel --host_jvm_args=-Xmx512m build \
--config=noaws \
--config=nogcp \
--config=nohdfs \
--config=noignite \
--config=nokafka \
--config=nonccl \
//tensorflow/tools/lib_package:libtensorflow
```
Prebuilt binaries are generated in **`bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz`**.

# ４．Install Tensorflow C API

```console:
$ tar -C /usr/local -xzf libtensorflow.tar.gz
```

# ５．Reference articles

- **[64-bit OS image creation repository for RaspberryPi3/4](https://github.com/drtyhlpr/rpi23-gen-image)**
- **[How to install Ubuntu 18.04 aarch64 (64bit) on RaspberryPi3](https://qiita.com/PINTO/items/4f3bca0629bc41f22b83)**