GCE×CentOS8 でJupyter環境を構築する

# GCE × CentOS8 で Jupyter 環境を構築する

タイトルのままですが、CentOS8 が使えたので、勉強がてら触ってみました。
基本的には Docker および docker-compose を使用してコンテナの Jupyter 環境を構築します。

## docker のインストール

どんな方法でもいいので、GCP 上でインスタンスを立ててください。
その際に CentOS8 のイメージにするのを忘れずに。

SSH でコンソールに繋いだところからはじめます。

まずは yum update の代わり。

```
[user@jupyter ~]$ sudo su -
```
```
[root@jupyter ~]# dnf -y update
CentOS-8 - AppStream                                                               1.5 MB/s | 5.6 MB     00:03
CentOS-8 - Base                                                                    2.0 MB/s | 5.3 MB     00:02
CentOS-8 - Extras                                                                  1.5 kB/s | 2.1 kB     00:01
Google Cloud SDK                                                                   5.9 MB/s |  27 MB     00:04
Google Compute Engine                                                              1.6 kB/s | 5.7 kB     00:03
Dependencies resolved.
===================================================================================================================
 Package                        Arch                 Version                  Repository                      Size
===================================================================================================================
Upgrading:
 google-cloud-sdk               noarch               266.0.0-1                google-cloud-sdk                35 M
Transaction Summary
===================================================================================================================
Upgrade  1 Package
Total download size: 35 M
Downloading Packages:
8ad4cecdf8d21424d0e49bca00836de86cb608625384064f5d6a0c73eae2df16-google-cloud-sdk-  12 MB/s |  35 MB     00:02
-------------------------------------------------------------------------------------------------------------------
Total                                                                               12 MB/s |  35 MB     00:02
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                           1/1
  Upgrading        : google-cloud-sdk-266.0.0-1.noarch                                                         1/2
  Running scriptlet: google-cloud-sdk-266.0.0-1.noarch                                                         1/2
  Running scriptlet: google-cloud-sdk-265.0.0-1.noarch                                                         2/2
  Cleanup          : google-cloud-sdk-265.0.0-1.noarch                                                         2/2
  Running scriptlet: google-cloud-sdk-265.0.0-1.noarch                                                         2/2
  Verifying        : google-cloud-sdk-266.0.0-1.noarch                                                         1/2
  Verifying        : google-cloud-sdk-265.0.0-1.noarch                                                         2/2
Upgraded:
  google-cloud-sdk-266.0.0-1.noarch
Complete!
```

installの前準備

```
[root@jupyter ~]# dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
```
```
[root@jupyter ~]# dnf repolist
Docker CE Stable - x86_64 3.0 kB/s | 3.5 kB 00:01
Google Cloud SDK 195 B/s | 454 B 00:02
Google Compute Engine 199 B/s | 454 B 00:02
repo id repo name status
AppStream CentOS-8 - AppStream 4,928
BaseOS CentOS-8 - Base 2,713
docker-ce-stable Docker CE Stable - x86_64 53
extras CentOS-8 - Extras 3
google-cloud-sdk Google Cloud SDK 948
google-compute-engine Google Compute Engine 6
```

なぜかそのままinstallができないので
別バージョンを指定

```
[root@jupyter ~]# dnf -y install wget
Last metadata expiration check: 0:02:03 ago on Fri 11 Oct 2019 01:56:03 AM UTC.
Dependencies resolved.
===================================================================================================================
Package Arch Version Repository Size
===================================================================================================================
Installing:
wget x86_64 1.19.5-7.el8_0.1 AppStream 734 k
Transaction Summary
===================================================================================================================
Install 1 Package
Total download size: 734 k
Installed size: 2.8 M
Downloading Packages:
wget-1.19.5-7.el8_0.1.x86_64.rpm 608 kB/s | 734 kB 00:01

---

Total 318 kB/s | 734 kB 00:02  
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : wget-1.19.5-7.el8_0.1.x86_64 1/1
Running scriptlet: wget-1.19.5-7.el8_0.1.x86_64 1/1
Verifying : wget-1.19.5-7.el8_0.1.x86_64 1/1
Installed:
wget-1.19.5-7.el8_0.1.x86_64  
Complete!
```
```
[root@jupyter ~]# wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.2-3.3.el7.x86_64.rpm
--2019-10-11 01:58:53-- https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.2-3.3.
el7.x86_64.rpm
Resolving download.docker.com (download.docker.com)... 13.249.87.50, 13.249.87.47, 13.249.87.20, ...
Connecting to download.docker.com (download.docker.com)|13.249.87.50|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 23159364 (22M) [binary/octet-stream]
Saving to: ‘containerd.io-1.2.2-3.3.el7.x86_64.rpm’
containerd.io-1.2.2-3.3.el7. 100%[=============================================>] 22.09M 86.2MB/s in 0.3s  
2019-10-11 01:58:53 (86.2 MB/s) - ‘containerd.io-1.2.2-3.3.el7.x86_64.rpm’ saved [23159364/23159364][root@jupyter ~]#
```
```
[root@jupyter ~]# dnf remove containerd.io
No match for argument: containerd.io
No packages marked for removal.
Dependencies resolved.
Nothing to do.
Complete!
```
```
[root@jupyter ~]# dnf install containerd.io-1.2.2-3.3.el7.x86_64.rpm
Last metadata expiration check: 0:01:55 ago on Fri 11 Oct 2019 02:26:59 AM UTC.
Dependencies resolved.
=============================================================================================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================================================================================
Installing:
containerd.io x86_64 1.2.2-3.3.el7 @commandline 22 M

# Transaction Summary

Install 1 Package

Total size: 22 M
Installed size: 88 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : containerd.io-1.2.2-3.3.el7.x86_64 1/1
Running scriptlet: containerd.io-1.2.2-3.3.el7.x86_64 1/1
Verifying : containerd.io-1.2.2-3.3.el7.x86_64 1/1

Installed:
containerd.io-1.2.2-3.3.el7.x86_64

Complete!
```
```
[root@jupyter ~]# dnf install -y docker-ce docker-ce-cli
Last metadata expiration check: 0:04:01 ago on Fri 11 Oct 2019 01:56:03 AM UTC.
Dependencies resolved.
===================================================================================================================
Package Arch Version Repository Size
===================================================================================================================
Installing:
docker-ce x86_64 3:19.03.3-3.el7 docker-ce-stable 24 M
docker-ce-cli x86_64 1:19.03.3-3.el7 docker-ce-stable 39 M
Installing dependencies:
container-selinux noarch 2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7 AppStream 43 k
libcgroup x86_64 0.41-19.el8 BaseOS 70 k
policycoreutils-python-utils noarch 2.8-16.1.el8 BaseOS 228 k
Enabling module streams:
container-tools rhel8  
Transaction Summary
===================================================================================================================
Install 5 Packages

Total download size: 64 M
Installed size: 273 M
Downloading Packages:
(1/5): libcgroup-0.41-19.el8.x86_64.rpm 62 kB/s | 70 kB 00:01  
(2/5): container-selinux-2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7.noarch.rpm 39 kB/s | 43 kB 00:01  
(3/5): policycoreutils-python-utils-2.8-16.1.el8.noarch.rpm 187 kB/s | 228 kB 00:01  
(4/5): docker-ce-19.03.3-3.el7.x86_64.rpm 41 MB/s | 24 MB 00:00  
(5/5): docker-ce-cli-19.03.3-3.el7.x86_64.rpm 45 MB/s | 39 MB 00:00

---

Total 6.4 MB/s | 64 MB 00:10  
warning: /var/cache/dnf/docker-ce-stable-091d8a9c23201250/packages/docker-ce-19.03.3-3.el7.x86_64.rpm: Header V4 RS
A/SHA512 Signature, key ID 621e9f35: NOKEY
Docker CE Stable - x86_64 1.4 kB/s | 1.6 kB 00:01  
Importing GPG key 0x621E9F35:
Userid : "Docker Release (CE rpm) <docker@docker.com>"
Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
From : https://download.docker.com/linux/centos/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : docker-ce-cli-1:19.03.3-3.el7.x86_64 1/5
Running scriptlet: docker-ce-cli-1:19.03.3-3.el7.x86_64 1/5
Installing : policycoreutils-python-utils-2.8-16.1.el8.noarch 2/5
Installing : container-selinux-2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7.noarch 3/5
Running scriptlet: container-selinux-2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7.noarch 3/5
Running scriptlet: libcgroup-0.41-19.el8.x86_64 4/5
Installing : libcgroup-0.41-19.el8.x86_64 4/5
Running scriptlet: libcgroup-0.41-19.el8.x86_64 4/5
Installing : docker-ce-3:19.03.3-3.el7.x86_64 5/5
Running scriptlet: docker-ce-3:19.03.3-3.el7.x86_64 5/5
Verifying : container-selinux-2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7.noarch 1/5
Verifying : libcgroup-0.41-19.el8.x86_64 2/5
Verifying : policycoreutils-python-utils-2.8-16.1.el8.noarch 3/5
Verifying : docker-ce-3:19.03.3-3.el7.x86_64 4/5
Verifying : docker-ce-cli-1:19.03.3-3.el7.x86_64 5/5
Installed:
docker-ce-3:19.03.3-3.el7.x86_64 docker-ce-cli-1:19.03.3-3.el7.x86_64  
 container-selinux-2:2.94-1.git1e99f1d.module_el8.0.0+58+91b614e7.noarch libcgroup-0.41-19.el8.x86_64  
 policycoreutils-python-utils-2.8-16.1.el8.noarch  
Complete!
```
```
[root@jupyter ~]# docker --version
Docker version 19.03.3, build a872fc2f86
```
```
[root@jupyter ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS   NAMES
```

入りましたね。


自動起動設定をしておく

```
[root@jupyter ~]# systemctl start docker
```
```
[root@jupyter ~]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
```

docker-composeは入っていない状態。

```
[root@jupyter ~]# docker-compose
-bash: docker-compose: command not found
```

dnfもない。

```
[root@jupyter ~]# dnf -y install docker-compose
Last metadata expiration check: 0:07:38 ago on Fri 11 Oct 2019 01:56:03 AM UTC.
No match for argument: docker-compose
Error: Unable to find a match
```


[公式の通り](https://docs.docker.com/compose/install/#install-compose)にいれる

```
[root@jupyter ~]# curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 617 0 617 0 0 2924 0 --:--:-- --:--:-- --:--:-- 2910
100 15.4M 100 15.4M 0 0 20.9M 0 --:--:-- --:--:-- --:--:-- 20.9M
```
```
[root@jupyter ~]# chmod +x /usr/local/bin/docker-compose
```
```
[root@jupyter ~]# ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
```
```
[root@jupyter ~]# docker-compose --version
docker-compose version 1.24.1, build 4667896b
```

うまく入った。


## Jupyterコンテナの準備
Dockerの準備に比べて、やることは少ないです。
juyterのDockerfileを設置したいディレクトリに移動し、
下記のリポジトリをクローンします。

```bash
dnf -y install git 
```

```bash
git clone https://github.com/hiroyannnn/jupyter.git
```

cloneするのが嫌な場合は、リポジトリ内のファイルをviなどで作成しても良いです。

cloneした場合、jupyterというディレクトリができるので、cdします。

```bash
cd jupyter
```

configファイルにパスフレーズを入れているので、
opensslなどでパスフレーズを作ってください。

`sha1:d4a8b1853fdf:(...省略...)fa181ec59716`
みたいなやつです。

```bash
# create your key
# and modify config file.
```

いれる場所はここ

```jupyter_notebook_config.py
c.NotebookApp.password = ''
```

Dockerを起動します。

```bash
docker-compose up
```

コンソールにログが出力されるので、
起動したっぽいログがでたら、
パブリックIP:8888 でアクセスできるはず。
暗号化したパスフレーズを入力してログインしてください。

データは`data`ディレクトリに入ります。

ついでに、ホスト側のdataは、
GCEの追加diskをマウントしておくと、データ量が増えた時も調整しやすいです。

## 参考
https://qiita.com/dora_2562/items/24691d3bec4c99c1d794
https://qiita.com/y4m3/items/c2703d4e131e05084b7b


## 余談
ちなみに一回下記でコマンドミスして

```bash
# 本来はこっち
# dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# 間違えてliで改行が入っちゃった
dnf config-manager --add-repo https://download.docker.com/li
```

repolistが失敗するようになりましたが

```
[root@jupyter ~]# dnf repolist
CentOS-8 - AppStream 1.8 kB/s | 4.3 kB 00:02
CentOS-8 - Base 1.6 kB/s | 3.8 kB 00:02
CentOS-8 - Extras 1.1 kB/s | 1.5 kB 00:01
created by dnf config-manager from https://download.docker.com/li 456 B/s | 550 B 00:01
Error: Failed to synchronize cache for repo 'download.docker.com_li'
```

そのファイルを消しちゃえばリカバリ可能でした。

```
[root@jupyter ~]# ls -l /etc/yum.repos.d/
total 56
-rw-r--r--. 1 root root 731 Aug 14 06:42 CentOS-AppStream.repo
-rw-r--r--. 1 root root 712 Aug 14 06:42 CentOS-Base.repo
-rw-r--r--. 1 root root 798 Aug 14 06:42 CentOS-centosplus.repo
-rw-r--r--. 1 root root 1320 Aug 14 06:42 CentOS-CR.repo
-rw-r--r--. 1 root root 668 Aug 14 06:42 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root 756 Aug 14 06:42 CentOS-Extras.repo
-rw-r--r--. 1 root root 338 Aug 14 06:42 CentOS-fasttrack.repo
-rw-r--r--. 1 root root 928 Aug 14 06:42 CentOS-Media.repo
-rw-r--r--. 1 root root 736 Aug 14 06:42 CentOS-PowerTools.repo
-rw-r--r--. 1 root root 1382 Aug 14 06:42 CentOS-Sources.repo
-rw-r--r--. 1 root root 74 Aug 14 06:42 CentOS-Vault.repo
-rw-r--r--. 1 root root 2424 Oct 11 01:43 docker-ce.repo
-rw-r--r--. 1 root root 145 Oct 11 01:43 download.docker.com_li.repo
-rw-r--r--. 1 root root 591 Oct 2 22:49 google-cloud.repo
```

```
[root@jupyter ~]# rm /etc/yum.repos.d/download.docker.com_li.repo
rm: remove regular file '/etc/yum.repos.d/download.docker.com_li.repo'? y
```

```
[root@jupyter ~]# dnf repolist
Docker CE Stable - x86_64 3.0 kB/s | 3.5 kB 00:01
Google Cloud SDK 195 B/s | 454 B 00:02
Google Compute Engine 199 B/s | 454 B 00:02
repo id repo name status
AppStream CentOS-8 - AppStream 4,928
BaseOS CentOS-8 - Base 2,713
docker-ce-stable Docker CE Stable - x86_64 53
extras CentOS-8 - Extras 3
google-cloud-sdk Google Cloud SDK 948
google-compute-engine Google Compute Engine 6
[root@jupyter ~]#
```