Alpine Linuxにnumpy, scipy, scikit-learn, pandasを入れた

# 概要

* Alpine LinuxにPythonを入れた。
* Pythonライブラリとして、numpy,scipy,scikit-learn,pandasを入れた。
* Dockerビルドして、コンテナ内でPythonライブラリをインポートできることを確認した。
* ~~基本的に最新版を使用したが、pandasは0.20.3を使用した(理由は後述)。~~ 

pandasの0.23.0を入れる回避策がわかったので追記した。

# はじめに

[Alpine Linux packages](https://pkgs.alpinelinux.org/packages)にはnumpy(py3-numpy)はあるけど、scipy,scikit-learn,pandasはない。
また、numpyも[PyPI](https://pypi.org/)の更新のほうが早いので、これらのライブラリをpipでインストールしてみる。

# 前提環境

2012年に買ったMacBookAirを使った。

* MacOS Sierra
* メモリ8GB

# 手順

* [Dockerのダウンロードページ(forMac)](https://store.docker.com/editions/community/docker-ce-desktop-mac)よりDocker-CEをダウンロードして、インストール。

* Dockerがインストールできたことを確認する。

```
# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
```

* [Alpine Linux公式のダウンロードページ](https://alpinelinux.org/downloads/)からベースイメージの元を入手。今回はMINI ROOT FILESYSTEMのX86_64をダウンロードした。

* ベースイメージを作る。

```
cat alpine-minirootfs-3.7.0-x86_64.tar.gz | docker import - alpine:3.7 
```

* DockerFileを書く。pipを実行するとその延長でビルドが実行されるため、ビルドで必要なものをapk addで先に入れておく必要がある。

```
FROM alpine:3.7

RUN apk --update-cache \
    add musl \
	linux-headers \
	gcc \
	g++ \
	make \
	gfortran \
	openblas-dev \
	python3 \
	python3-dev

RUN pip3 install --upgrade pip 
RUN pip3 install numpy \
	scipy  \
	scikit-learn \ 
	'pandas<0.21.0'
```

* Dockerビルドを実行する。実行すると必要なパッケージがネットからダウンロードされる。私の環境だとそこそこ時間がかかり、特にscipyのビルドはしばらく待つ必要がある。

```
docker build -t python-test:latest .
Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM alpine:3.7
 ---> 70cb411a7a13
Step 2/4 : RUN apk --update-cache     add musl 	linux-headers 	gcc 	g++ 	make 	gfortran 	openblas-dev 	python3 	python3-dev
 ---> Running in f429cc126e22
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/37) Upgrading musl (1.1.18-r2 -> 1.1.18-r3)
(2/37) Installing libgcc (6.4.0-r5)
(3/37) Installing libstdc++ (6.4.0-r5)
(4/37) Installing binutils-libs (2.28-r3)
(5/37) Installing binutils (2.28-r3)
(6/37) Installing gmp (6.1.2-r1)
(7/37) Installing isl (0.18-r0)
(8/37) Installing libgomp (6.4.0-r5)
(9/37) Installing libatomic (6.4.0-r5)
(10/37) Installing pkgconf (1.3.10-r0)
(11/37) Installing mpfr3 (3.1.5-r1)
(12/37) Installing mpc1 (1.0.3-r1)
(13/37) Installing gcc (6.4.0-r5)
(14/37) Installing musl-dev (1.1.18-r3)
(15/37) Installing libc-dev (0.7.1-r0)
(16/37) Installing g++ (6.4.0-r5)
(17/37) Installing libquadmath (6.4.0-r5)
(18/37) Installing libgfortran (6.4.0-r5)
(19/37) Installing gfortran (6.4.0-r5)
(20/37) Upgrading musl-utils (1.1.18-r2 -> 1.1.18-r3)
(21/37) Installing linux-headers (4.4.6-r2)
(22/37) Installing make (4.2.1-r0)
(23/37) Installing openblas-ilp64 (0.2.19-r3)
(24/37) Installing openblas (0.2.19-r3)
(25/37) Installing openblas-dev (0.2.19-r3)
(26/37) Installing libbz2 (1.0.6-r6)
(27/37) Installing expat (2.2.5-r0)
(28/37) Installing libffi (3.2.1-r4)
(29/37) Installing gdbm (1.13-r1)
(30/37) Installing xz-libs (5.2.3-r1)
(31/37) Installing ncurses-terminfo-base (6.0_p20171125-r0)
(32/37) Installing ncurses-terminfo (6.0_p20171125-r0)
(33/37) Installing ncurses-libs (6.0_p20171125-r0)
(34/37) Installing readline (7.0.003-r0)
(35/37) Installing sqlite-libs (3.21.0-r0)
(36/37) Installing python3 (3.6.3-r9)
(37/37) Installing python3-dev (3.6.3-r9)
Executing busybox-1.27.2-r6.trigger
OK: 445 MiB in 46 packages
Removing intermediate container f429cc126e22
 ---> 17d9b2245094
Step 3/4 : RUN pip3 install --upgrade pip
 ---> Running in df491784f1c5
Collecting pip
  Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
Successfully installed pip-10.0.1
Removing intermediate container df491784f1c5
 ---> 5befda720b1d
Step 4/4 : RUN pip3 install numpy 	scipy  	scikit-learn 	'pandas<0.21.0'
 ---> Running in 4c92cecf4978
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/0b/66/86185402ee2d55865c675c06a5cfef742e39f4635a4ce1b1aefd20711c13/numpy-1.14.2.zip (4.9MB)
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/bd/f4/3882758754dc083fea6ea66a6e8ceef55e7df173d06a12a074612958800f/scipy-1.0.1.tar.gz (15.5MB)
Collecting scikit-learn
  Downloading https://files.pythonhosted.org/packages/f5/2c/5edf2488897cad4fb8c4ace86369833552615bf264460ae4ef6e1f258982/scikit-learn-0.19.1.tar.gz (9.5MB)
Collecting pandas<0.21.0
  Downloading https://files.pythonhosted.org/packages/ee/aa/90c06f249cf4408fa75135ad0df7d64c09cf74c9870733862491ed5f3a50/pandas-0.20.3.tar.gz (10.4MB)
Collecting python-dateutil>=2 (from pandas<0.21.0)
  Downloading https://files.pythonhosted.org/packages/0c/57/19f3a65bcf6d5be570ee8c35a5398496e10a0ddcbc95393b2d17f86aaaf8/python_dateutil-2.7.2-py2.py3-none-any.whl (212kB)
Collecting pytz>=2011k (from pandas<0.21.0)
  Downloading https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl (510kB)
Collecting six>=1.5 (from python-dateutil>=2->pandas<0.21.0)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Installing collected packages: numpy, scipy, scikit-learn, six, python-dateutil, pytz, pandas
  Running setup.py install for numpy: started
    Running setup.py install for numpy: still running...
    Running setup.py install for numpy: still running...
    Running setup.py install for numpy: finished with status 'done'
  Running setup.py install for scipy: started
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: still running...
    Running setup.py install for scipy: finished with status 'done'
  Running setup.py install for scikit-learn: started
    Running setup.py install for scikit-learn: still running...
    Running setup.py install for scikit-learn: still running...
    Running setup.py install for scikit-learn: finished with status 'done'
  Running setup.py install for pandas: started
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: finished with status 'done'
Successfully installed numpy-1.14.2 pandas-0.20.3 python-dateutil-2.7.2 pytz-2018.4 scikit-learn-0.19.1 scipy-1.0.1 six-1.11.0
Removing intermediate container 4c92cecf4978
 ---> 84e2d58c2a98
Successfully built 84e2d58c2a98
Successfully tagged python-test:latest
```

* Dockerコンテナを起動してコンテナ内に入る。

```
docker run -it python-test:latest /bin/sh
```

* pipで入れたパッケージをインポートしてみてエラーが出ないことを確認する。

```
/ # python3
Python 3.6.3 (default, Nov 21 2017, 14:55:19) 
[GCC 6.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import sklearn
>>> import pandas
>>> import scipy
```

# インストール済みパッケージ一覧(apk)

```
# apk info
busybox
alpine-baselayout
alpine-keys
libressl2.6-libcrypto
libressl2.6-libssl
zlib
apk-tools
scanelf
libc-utils
musl
libgcc
libstdc++
binutils-libs
binutils
gmp
isl
libgomp
libatomic
pkgconf
mpfr3
mpc1
gcc
musl-dev
libc-dev
g++
libquadmath
libgfortran
gfortran
musl-utils
linux-headers
make
openblas-ilp64
openblas
openblas-dev
libbz2
expat
libffi
gdbm
xz-libs
ncurses-terminfo-base
ncurses-terminfo
ncurses-libs
readline
sqlite-libs
python3
python3-dev
```

# インストール済みパッケージ一覧(pip)

```
# pip3 list
Package         Version
--------------- -------
numpy           1.14.2 
pandas          0.20.3 
pip             10.0.1 
python-dateutil 2.7.2  
pytz            2018.4 
scikit-learn    0.19.1 
scipy           1.0.1  
setuptools      28.8.0 
six             1.11.0 
```

# pandasの0.21.0以降のインストール

以下のようなエラーになったので、今回は0.21.0より前のバージョンを入れて回避した。ちなみに、以下は0.21.0を指定して実行した場合のエラーだが最新版でも同じエラーメッセージ。

```
# pip3 install 'pandas==0.21.0'
Collecting pandas==0.21.0
  Downloading https://files.pythonhosted.org/packages/84/4c/622301f99725c6da76ae8079cb6b9091d9098098bd6b4eac47fb502b32f4/pandas-0.21.0.tar.gz (11.3MB)
    100% |████████████████████████████████| 11.3MB 565kB/s 
  Could not find a version that satisfies the requirement Cython (from versions: )
No matching distribution found for Cython 
```

以下の記事が関係あるのかも知れないが、見れていない。
[Pandas installation problems with pip version 10](https://github.com/pandas-dev/pandas/issues/20775)

# pandas最新版のインストール

[pip 10 breaks pandas on alpine linux](https://github.com/pypa/pip/issues/5299)に、```--no-build-isolation```をつけて回避する方法が乗っていたので試してみた。

```
Step 7/8 : RUN pip3 install --no-cache-dir pandas --no-build-isolation
 ---> Running in f60bf6bead58
Collecting pandas
  Downloading https://files.pythonhosted.org/packages/45/12/1e1ba99fb65df9f7f3724d3232feef35cc044d18604d57492d561e90219f/pandas-0.23.0.tar.gz (13.1MB)
Collecting python-dateutil>=2.5.0 (from pandas)
  Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB)
Collecting pytz>=2011k (from pandas)
  Downloading https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl (510kB)
Requirement already satisfied: numpy>=1.9.0 in /usr/lib/python3.6/site-packages (from pandas) (1.14.3)
Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, pytz, pandas
  Running setup.py install for pandas: started
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: still running...
    Running setup.py install for pandas: finished with status 'done'
Successfully installed pandas-0.23.0 python-dateutil-2.7.3 pytz-2018.4 six-1.11.0
Removing intermediate container f60bf6bead58
 ---> 457c8f8f68ca
```

バージョンを確認。

```
/ # python3
Python 3.6.3 (default, Nov 21 2017, 14:55:19) 
[GCC 6.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> print(pd.__version__)
0.23.0
```


# 今後

現在はビルドに必要なものが全部残った状態となっていて、とにかく軽量なAlpine Linuxの利点を消している状態なので、必要なもの以外を消して(apk del)サイズを見比べたい。