kubernetes-incubator/cri-o チュートリアル


参考 : [https://github.com/kubernetes-incubator/cri-o/blob/master/tutorial.md](https://github.com/kubernetes-incubator/cri-o/blob/master/tutorial.md)


Open Container Initiativeの実装に基づくKubernetes Container Runtime Interface(CRI) - cri-oをインストールし、Pod内にRedis Serverを起動する。

cri-oがフォーカスする機能は以下の通り。

- Support multiple image formats including the existing Docker image format
- Support for multiple means to download images including trust & image verification
- Container image management (managing image layers, overlay filesystems, etc)
- Container process lifecycle management
- Monitoring and logging required to satisfy the CRI
- Resource isolation as required by the CRI

以下のコンポーネントをインストールして利用する。

- ocid - The implementation of the Kubernetes CRI (Container Runtime Interface), which manages Pods.
- ocic - The ocid client for testing.
- [cni - The Container Network Interface](https://github.com/containernetworking/cni)
- [runc - The OCI runtime to launch the container](https://github.com/opencontainers/runc)

## 必要なパッケージのインストール

```
$ sudo apt update
$ sudo apt install -y btrfs-tools ¥
   libassuan-dev ¥
   libdevmapper-dev ¥
   libglib2.0-dev ¥
   libc6-dev ¥
   libgpgme11-dev ¥
   libgpg-error-dev ¥
   libseccomp-dev ¥
   libselinux1-dev ¥
   pkg-config
$ sudo apt install -y libapparmor-dev
$ sudo apt install -y golang
```

## runcのインストール

ビルド済みのバイナリをダウンロードしてインストール

```
$ wget https://github.com/opencontainers/runc/releases/download/v1.0.0-rc2/runc-linux-amd64
$ sudo mv runc-linux-amd64 /usr/bin/runc && sudo chmod +x /usr/bin/runc
$ runc -version
runc version 1.0.0-rc2
commit: c91b5bea4830a57eac7882d7455d59518cdf70ec
spec: 1.0.0-rc2-dev
```

## cri-oのビルドとインストール

cri-o(ocid/ocic)をビルドしてインストール。

```
$ mkdir ~/go
$ export GOPATH=~/go
$ go get -d github.com/kubernetes-incubator/cri-o
package github.com/kubernetes-incubator/cri-o: no buildable Go source files in /home/ubuntu/go/src/github.com/kubernetes-incubator/cri-o
$ cd $GOPATH/src/github.com/kubernetes-incubator/cri-o
$ make install.tools
$ make
$ sudo GOPATH=$GOPATH make install
install -D -m 755 ocid /usr/local/bin/ocid
install -D -m 755 ocic /usr/local/bin/ocic
install -D -m 755 kpod /usr/local/bin/kpod
install -D -m 755 conmon/conmon /usr/local/libexec/ocid/conmon
install -D -m 755 pause/pause /usr/local/libexec/ocid/pause
install -d -m 755 /usr/local/share/man/man1
install -d -m 755 /usr/local/share/man/man5
install -d -m 755 /usr/local/share/man/man8
install -m 644 docs/kpod.1 docs/kpod-launch.1 -t /usr/local/share/man/man1
install -m 644 docs/ocid.conf.5 -t /usr/local/share/man/man5
install -m 644 docs/ocid.8 -t /usr/local/share/man/man8
$ sudo make install.config
install -D -m 644 ocid.conf /etc/ocid/ocid.conf
install -D -m 644 seccomp.json /etc/ocid/seccomp.json
```

## cniのインストール

cniをビルドして /opt/cni/bin にインストールする。/etc/ocid/ocid.confで以下の通り設定されているので、バイナリと設定ファイルをocidの設定ファイルにもとづいて配置する。

```
[ocid.network]

# network_dir is is where CNI network configuration
# files are stored.
network_dir = "/etc/cni/net.d/"

# plugin_dir is is where CNI plugin binaries are stored.
plugin_dir = "/opt/cni/bin/"
```

```
$ go get -d github.com/containernetworking/cni
package github.com/containernetworking/cni: no buildable Go source files in /home/ubuntu/go/src/github.com/containernetworking/cni
$ cd $GOPATH/src/github.com/containernetworking/cni
$ ./build.sh
Building API
Building reference CLI
Building plugins
   flannel
   tuning
   bridge
   ipvlan
   loopback
   macvlan
   ptp
   dhcp
   host-local
   noop
$ sudo mkdir -p /opt/cni/bin
$ sudo cp bin/* /opt/cni/bin/
```

### cniの設定ファイルの作成

cri-oリポジトリのcontrib/cniにある設定ファイルを/etc/cni/net.dにコピーする。

```
$ sudo mkdir -p /etc/cni/net.d
$ sudo cp $GOPATH/src/github.com/kubernetes-incubator/cri-o/contrib/cni/99-loopback.conf /etc/cni/net.d/
$ sudo cp $GOPATH/src/github.com/kubernetes-incubator/cri-o/contrib/cni/10-ocid-bridge.conf /etc/cni/net.d/
```

## policy.jsonのコピー

ocidのsignature policyを制御するpolicy.jsonを作成する。

```
$ sudo mkdir /etc/containers
$ sudo cp $GOPATH/src/github.com/kubernetes-incubator/cri-o/test/policy.json /etc/containers/
$ cat /etc/containers/policy.json
{
    "default": [
        {
            "type": "insecureAcceptAnything"
        }
    ]
}
```

## ocid systeam daemonの設定

```
$ sudo sh -c 'echo "[Unit]
Description=OCI-based implementation of Kubernetes Container Runtime Interface
Documentation=https://github.com/kubernetes-incubator/cri-o

[Service]
ExecStart=/usr/local/bin/ocid --debug
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target" > /etc/systemd/system/ocid.service'
$ sudo systemctl daemon-reload
$ sudo systemctl enable ocid
$ sudo systemctl start ocid
$ sudo ocic runtimeversion
VersionResponse: Version: 0.1.0, RuntimeName: runc, RuntimeVersion: 1.0.0-rc2, RuntimeApiVersion: v1alpha1
```

## Podの起動と確認

Podを起動する。

```
$ cd $GOPATH/src/github.com/kubernetes-incubator/cri-o
$ sudo ocic pod run --config test/testdata/sandbox_config.json
5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
```

起動したPodを確認。

```
$ sudo ocic pod status --id 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
ID: 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
Name: podsandbox1
UID: redhat-test-ocid
Namespace: redhat.test.ocid
Attempt: 1
Status: SANDBOX_READY
Created: 2017-04-11 21:42:29.998485391 +0000 UTC
Network namespace: /var/run/netns/cni-0ece8226-42d9-e1ae-e4c0-b4744c93a994
IP Address: 10.88.0.2
Labels:
	group -> test
Annotations:
	owner -> hmeng
	security.alpha.kubernetes.io/seccomp/pod -> unconfined
	security.alpha.kubernetes.io/sysctls -> kernel.shm_rmid_forced=1,net.ipv4.ip_local_port_range=1024 65000
	security.alpha.kubernetes.io/unsafe-sysctls -> kernel.msgmax=8192
```

Podの作成によりcni0ブリッジとvethが作成される。

```
$ brctl show
bridge name	bridge id		STP enabled	interfaces
cni0		8000.0a580a580001	no		veth314b177f
```

```
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:2d:8f:cf:d6:cd brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global enp0s3
       valid_lft forever preferred_lft forever
    inet6 fe80::2d:8fff:fecf:d6cd/64 scope link
       valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:f4:b0:67 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.60/24 brd 192.168.1.255 scope global enp0s8
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fef4:b067/64 scope link
       valid_lft forever preferred_lft forever
4: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0a:58:0a:58:00:01 brd ff:ff:ff:ff:ff:ff
    inet 10.88.0.1/16 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::a431:1cff:fe89:a1a6/64 scope link
       valid_lft forever preferred_lft forever
5: veth314b177f@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni0 state UP group default
    link/ether d2:c8:9f:03:7d:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::d0c8:9fff:fe03:7dab/64 scope link
       valid_lft forever preferred_lft forever
```

## Pod内でコンテナを起動する

### コンテナイメージのpull

docker hubからイメージをpullする。

```
$ sudo ocic image pull redis:latest
$ sudo ocic image list
ID: 63455cefc90d9ca403e00ac1d545196dfbdb3b3872b66e094d5a2394310e3c8b
Tag: docker.io/kubernetes/pause:latest
Tag: kubernetes/pause
ID: 402d05ced4b88a18c0c83e92e4bd63b01fe6caf85d8eb0d220d850fcc98d55cd
Tag: docker.io/library/redis:latest
Tag: redis:latest
```

### コンテナの作成

作成したPod内でredisコンテナを起動する。

```
$ sudo ocic ctr create --pod 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44 --config test/testdata/container_redis.json
6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
```

作成したコンテナの確認

```
$ sudo ocic ctr list
ID: 6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
Pod: 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
Name: podsandbox1-redis
Attempt: 0
Status: CONTAINER_CREATED
Image: docker://redis:latest
Created: 2017-04-11 21:52:43.845269865 +0000 UTC
Labels:
	tier -> backend
Annotations:
	pod -> podsandbox1

```

### コンテナの起動

```
$ sudo ocic ctr start --id 6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
```

起動の確認

```
$ sudo ocic ctr status --id 6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
ID: 6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
Name: podsandbox1-redis
Attempt: 0
Status: CONTAINER_RUNNING
Created: 2017-04-11 21:52:43.845269865 +0000 UTC
Started: 2017-04-11 21:53:57.117738023 +0000 UTC
Finished: 1970-01-01 00:00:00 +0000 UTC
Exit Code: 0
```


### Redisコンテナの起動確認

podのアドレスを確認して、telnetコマンドで接続確認

```
$ sudo ocic pod status --id 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
ID: 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
Name: podsandbox1
UID: redhat-test-ocid
Namespace: redhat.test.ocid
Attempt: 1
Status: SANDBOX_READY
Created: 2017-04-11 21:42:29.998485391 +0000 UTC
Network namespace: /var/run/netns/cni-0ece8226-42d9-e1ae-e4c0-b4744c93a994
IP Address: 10.88.0.2
Labels:
	group -> test
Annotations:
	owner -> hmeng
	security.alpha.kubernetes.io/seccomp/pod -> unconfined
	security.alpha.kubernetes.io/sysctls -> kernel.shm_rmid_forced=1,net.ipv4.ip_local_port_range=1024 65000
	security.alpha.kubernetes.io/unsafe-sysctls -> kernel.msgmax=8192

$ telnet 10.88.0.2 6379
Trying 10.88.0.2...
Connected to 10.88.0.2.
Escape character is '^]'.
MONITOR
+OK
^]quit

telnet> quit
Connection closed.
```

## コンテナとPodの削除

```
$ sudo ocic ctr remove --id 6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
6ba21b607a728262e5a4950cb2dbbf780ef3e341d6bcaa39e7f0243923bf4971
$ sudo ocic pod remove --id 5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
5c17c4c3c43263569f5075a21574d35f226be9c4ec82f82de3c2d9e14c6adf44
$ sudo ocic pod list
$ sudo ocic ctr list
```