Intel Edge Insights for Industrial (EII)の動画解析コンテナアプリケーションをOpenShiftへdeployしてみた

# はじめに

本記事は、下記の過去エントリで構築したOpenShiftクラスタへ、Intel&reg; Edge Insights for Industrial (以降、EII) のコンテナアプリケーションをdeployしてみます。

OpenShiftのインストール、コンテナレジストリの作成、EIIのコンテナイメージのビルド方法は以下の記事を参照して下さい。

https://qiita.com/ydo/items/874c7139119a17a4ecac

https://qiita.com/ydo/items/95fadeeff44c45b4f753

https://qiita.com/ydo/items/fc139e1e8d593f35590d


# 1. QuayレジストリにてEIIイメージのレポジトリを作成

QuayレジストリのWebコンソールにて`Create New Repository`をクリックし、`openedgeinsights`という名前のレジストリをprivateで作成します。

![1639534946058.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/559968/25679861-68f1-d7ae-c9d3-2c52577831c6.png)

# 2. EIIのコンテナイメージのpush

[この記事](https://qiita.com/ydo/items/fc139e1e8d593f35590d)にて構築したEIIの開発環境へアクセスします。

```bash
$ ssh -i <KEY> ubuntu@<作成したインスタンスのグローバルIPアドレス>
```

Quayレジストリのドメイン([この記事参照](https://qiita.com/ydo/items/95fadeeff44c45b4f753))へ`docker login`できることを確認します。



```bash
$ docker login QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN
Login Succeeded
```

以下のコンテナイメージを対象に、`docker tag`でイメージタグをつけます。Quay側へは`2.6.2`でpushします。

- ia_video_ingestion:2.6.1
- ia_video_analytics:2.6.1
- ia_etcd_ui:2.6.1
- ia_etcd:2.6.1
- ia_etcd_provision:2.6.1

```bash
$ sudo docker tag openedgeinsights/コンテナイメージ名:2.6.1 QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/コンテナイメージ名:2.6.2
```

`docker push`します。

```bash
$ sudo docker push QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/<コンテナイメージ名>:2.6.2
```

# 3. Project eiiを作成

```
$ oc new-project eii
```

# 4.DeploymentConfigのマニフェストを作成

5つの`DeploymentConfig`と1つの`ConfigMap`、`Route` 1つをdeployします。


```bash
├── eii-deploy
│   ├── dc-etcd-ui.yaml
│   ├── dc-va.yaml
│   ├── dc-vi.yaml
│   ├── dc-web.yaml
│   └── route.yaml
├── eii-provision
│   ├── dc-etcd.yaml
│   └──　　etcd-config.yaml
```

ディレクトリを作成します。

```bash
$ mkdir eii-provision
$ mkdir eii-deploy
```

## 4-1. etcd(DeploymentConfig)

```bash
$ vi eii-provision/dc-etcd.yaml
```

```yaml:eii-provision/dc-etcd.yaml
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: etcd
spec:
  replicas: 1
  selector:
    deployment-config.name: etcd
  template:
    metadata:
      labels:
        deployment-config.name: etcd
    spec:
      containers:
      - env:
        - name: AppName
          value: etcd
        - name: DEV_MODE
          value: "true"
        - name: ETCD_CLIENT_PORT
          value: "8379"
        - name: ETCD_PEER_PORT
          value: "8380"
        - name: ETCD_NAME
          value: leader
        - name: ETCD_INITIAL_CLUSTER_STATE
          value: new
        - name: ETCDCTL_API
          value: "3"
        - name: ETCD_DATA_DIR
          value: /EII/etcd/data/
        image: QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/ia_etcd:2.6.2
        imagePullPolicy: IfNotPresent
        name: etcd
      - args:
        - -c
        - |
          cd /EII/etcd/ && python3 etcd_provision.py docker-compose.yml && while true; do sleep 50000; done
        command:
        - /bin/sh
        env:
        - name: DEV_MODE
          value: "true"
        - name: ETCDCTL_API
          value: "3"
        - name: provision_mode
          value: k8s
        - name: ETCD_HOST
          value: etcd
        - name: ETCD_CLIENT_PORT
          value: "8379"
        - name: ETCD_PREFIX
        - name: ETCD_ROOT_PASSWORD
          value: eii123
        image: QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/ia_etcd_provision:2.6.2
        imagePullPolicy: IfNotPresent
        name: etcd-provision
        volumeMounts:
        - mountPath: /EII/etcd/config/eii_config.json
          name: eii-config
          subPath: eii_config.json
        - mountPath: /EII/etcd/docker-compose.yml
          name: docker-compose
          subPath: docker-compose.yml
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      volumes:
      - configMap:
          name: etcd-config
        name: eii-config
      - configMap:
          name: etcd-config
        name: docker-compose
      imagePullSecrets:
      - name: quay-registry-mysecret
      securityContext:
        runAsUser: 5315
  test: false
  triggers: null
---
apiVersion: v1
kind: Service
metadata:
  name: etcd
spec:
  ports:
  - name: tcp
    port: 8379
    protocol: TCP
    targetPort: 8379
  selector:
    deployment-config.name: etcd
  sessionAffinity: None
```

## 4-2. etcd-config(ConfigMap)

```bash
$ vi eii-provision/etcd-config.yaml
```

```yaml:eii-provision/etcd-config.yaml
apiVersion: v1
data:
  docker-compose.yml: |-
    version: '3.6'
    services:

      ia_etcd_ui:
        environment:
          AppName: "EtcdUI"
          CertType: "pem"

      ia_video_ingestion:
        environment:
          AppName: "VideoIngestion"
          CertType: "zmq"

      ia_video_analytics:
        environment:
          AppName: "VideoAnalytics"
          CertType: "zmq"

      ia_web_visualizer:
        environment:
          AppName: "WebVisualizer"
          CertType: "zmq,pem"

  eii_config.json: |-
    {
        "/EtcdUI/config": {},
        "/EtcdUI/interfaces": {},
        "/GlobalEnv/": {
            "C_LOG_LEVEL": "INFO",
            "ETCD_KEEPER_PORT": "7070",
            "GO_LOG_LEVEL": "INFO",
            "GO_VERBOSE": "0",
            "PY_LOG_LEVEL": "INFO"
        },
        "/VideoAnalytics/config": {
            "encoding": {
                "level": 95,
                "type": "jpeg"
            },
            "max_workers": 4,
            "queue_size": 10,
            "udfs": [
                {
                    "device": "CPU",
                    "model_bin": "common/video/udfs/python/pcb/ref/model_2.bin",
                    "model_xml": "common/video/udfs/python/pcb/ref/model_2.xml",
                    "name": "pcb.pcb_classifier",
                    "ref_config_roi": "common/video/udfs/python/pcb/ref/roi_2.json",
                    "ref_img": "common/video/udfs/python/pcb/ref/ref.png",
                    "type": "python"
                }
            ]
        },
        "/VideoAnalytics/interfaces": {
            "Publishers": [
                {
                    "AllowedClients": [
                        "*"
                    ],
                    "EndPoint": "0.0.0.0:65013",
                    "Name": "default",
                    "Topics": [
                        "camera1_stream_results"
                    ],
                    "Type": "zmq_tcp"
                }
            ],
            "Subscribers": [
                {
                    "EndPoint": "/EII/sockets",
                    "Name": "default",
                    "PublisherAppName": "VideoIngestion",
                    "Topics": [
                        "camera1_stream"
                    ],
                    "Type": "zmq_ipc",
                    "zmq_recv_hwm": 50
                }
            ]
        },
        "/VideoIngestion/config": {
            "encoding": {
                "level": 95,
                "type": "jpeg"
            },
            "ingestor": {
                "loop_video": true,
                "pipeline": "./test_videos/pcb_d2000.avi",
                "poll_interval": 0.2,
                "queue_size": 10,
                "type": "opencv"
            },
            "max_workers": 4,
            "sw_trigger": {
                "init_state": "running"
            },
            "udfs": [
                {
                    "n_left_px": 1000,
                    "n_right_px": 1000,
                    "n_total_px": 300000,
                    "name": "pcb.pcb_filter",
                    "scale_ratio": 4,
                    "training_mode": "false",
                    "type": "python"
                }
            ]
        },
        "/VideoIngestion/interfaces": {
            "Publishers": [
                {
                    "AllowedClients": [
                        "VideoAnalytics",
                        "Visualizer",
                        "WebVisualizer",
                        "TLSRemoteAgent",
                        "RestDataExport"
                    ],
                    "EndPoint": "/EII/sockets",
                    "Name": "default",
                    "Topics": [
                        "camera1_stream"
                    ],
                    "Type": "zmq_ipc"
                }
            ],
            "Servers": [
                {
                    "AllowedClients": [
                        "*"
                    ],
                    "EndPoint": "0.0.0.0:64013",
                    "Name": "default",
                    "Type": "zmq_tcp"
                }
            ]
        },
        "/Visualizer/config": {
            "draw_results": "true",
            "labels": {
                "camera1_stream_results": {
                    "0": "MISSING",
                    "1": "SHORT"
                },
                "gva_safety_gear_stream_results": {
                    "1": "safety_helmet",
                    "2": "safety_jacket",
                    "3": "Safe",
                    "4": "Violation"
                },
                "native_safety_gear_stream_results": {
                    "1": "safety_helmet",
                    "2": "safety_jacket",
                    "3": "Safe",
                    "4": "Violation"
                },
                "py_safety_gear_stream_results": {
                    "1": "safety_helmet",
                    "2": "safety_jacket",
                    "3": "Safe",
                    "4": "Violation"
                }
            },
            "save_image": "false"
        },
        "/Visualizer/interfaces": {
            "Subscribers": [
                {
                    "EndPoint": "ia_video_analytics:65013",
                    "Name": "default",
                    "PublisherAppName": "VideoAnalytics",
                    "Topics": [
                        "camera1_stream_results"
                    ],
                    "Type": "zmq_tcp"
                }
            ]
        },
        "/WebVisualizer/config": {
            "dev_port": 5001,
            "draw_results": "true",
            "labels": {
                "camera1_stream_results": {
                    "0": "MISSING",
                    "1": "SHORT"
                },
                "gva_safety_gear_stream_results": {
                    "1": "safety_helmet",
                    "2": "safety_jacket",
                    "3": "Safe",
                    "4": "Violation"
                },
                "native_safety_gear_stream_results": {
                    "1": "safety_helmet",
                    "2": "safety_jacket",
                    "3": "Safe",
                    "4": "Violation"
                },
                "py_safety_gear_stream_results": {
                    "1": "safety_helmet",
                    "2": "safety_jacket",
                    "3": "Safe",
                    "4": "Violation"
                }
            },
            "password": "admin@123",
            "port": 5000,
            "username": "admin"
        },
        "/WebVisualizer/interfaces": {
            "Subscribers": [
                {
                    "EndPoint": "ia_video_analytics:65013",
                    "Name": "default",
                    "PublisherAppName": "VideoAnalytics",
                    "Topics": [
                        "camera1_stream_results"
                    ],
                    "Type": "zmq_tcp"
                }
            ]
        }
    }
kind: ConfigMap
metadata:
  labels:
    app: eii-provision 
  name: etcd-config
```

## 4-3. etcd-ui(DeploymentConfig)

```bash
$ vi eii-deploy/dc-etcd-ui.yaml
```

```yaml:eii-deploy/dc-etcd-ui.yaml
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: etcd-ui
spec:
  replicas: 1
  selector:
    deployment-config.name: etcd-ui
  template:
    metadata:
      labels:
        deployment-config.name: etcd-ui
    spec:
      containers:
      - env:
        - name: AppName
          value: EtcdUI
        - name: DEV_MODE
          value: "true"
        - name: NGINX_PORT
          value: "7071"
        - name: ETCD_HOST
          value: etcd
        - name: ETCD_CLIENT_PORT
          value: "8379"
        - name: PROVISION_MODE
        - name: ETCD_PREFIX
        - name: ETCD_ENDPOINT
          value: etcd:8379
        - name: ETCD_USER
          value: root
        image: QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/ia_etcd_ui:2.6.2
        imagePullPolicy: IfNotPresent
        name: ia-etcd-ui
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      imagePullSecrets:
      - name: quay-registry-mysecret
      securityContext:
        runAsUser: 5315
---
apiVersion: v1
kind: Service
metadata:
  name: etcd-ui
spec:
  ports:
  - port: 7071
    protocol: TCP
    targetPort: 7071
  selector:
    deployment-config.name: etcd-ui
```

## 4-4. Video Analytics(DeploymentConfig)

```bash
$ vi eii-deploy/dc-va.yaml
```

```yaml:eii-deploy/dc-va.yaml
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: videoanalytics 
spec:
  replicas: 1
  selector:
    deployment-config.name: videoanalytics
  template:
    metadata:
      labels:
        deployment-config.name: videoanalytics
    spec:
      containers:
      - env:
        - name: AppName
          value: VideoAnalytics
        - name: DEV_MODE
          value: "true"
        - name: PROFILING_MODE
          value: "false"
        - name: ETCD_HOST
          value: etcd
        - name: ETCD_CLIENT_PORT
          value: "8379"
        - name: SUBSCRIBER_ENDPOINT
          value: videoingestion:65012
        - name: SUBSCRIBER_TYPE
          value: zmq_tcp
        - name: PUBLISHER_ENDPOINT
          value: 0.0.0.0:65013
        - name: PUBLISHER_TYPE
          value: zmq_tcp
        image: QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/ia_video_analytics:2.6.2
        imagePullPolicy: IfNotPresent
        name: ia-video-analytics
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      imagePullSecrets:
      - name: quay-registry-mysecret
      securityContext:
        runAsUser: 5315
---
apiVersion: v1
kind: Service
metadata:
  name: videoanalytics
spec:
  ports:
  - name: tcp
    port: 65013
    protocol: TCP
    targetPort: 65013
  selector:
    deployment-config.name: videoanalytics
```

## 4-5. Video Ingestion(DeploymentConfig)

```bash
$ vi eii-deploy/dc-vi.yaml
```

```yaml:eii-deploy/dc-vi.yaml
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: videoingestion 
spec:
  replicas: 1
  selector:
    deployment-config.name: videoingestion 
  template:
    metadata:
      labels:
        deployment-config.name: videoingestion
    spec:
      containers:
      - env:
        - name: AppName
          value: VideoIngestion
        - name: DEV_MODE
          value: "true"
        - name: PROFILING_MODE
          value: "false"
        - name: ETCD_HOST
          value: etcd
        - name: ETCD_CLIENT_PORT
          value: "8379"
        - name: ETCD_PREFIX
        - name: GENICAM
          value: Matrix_Vision
        - name: PUBLISHER_ENDPOINT
          value: 0.0.0.0:65012
        - name: PUBLISHER_TYPE
          value: zmq_tcp
        - name: SERVER_ENDPOINT
          value: 0.0.0.0:64013
        - name: SERVER_TYPE
          value: zmq_tcp
        image: QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/ia_video_ingestion:2.6.2
        imagePullPolicy: IfNotPresent
        name: ia-video-ingestion
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      imagePullSecrets:
      - name: quay-registry-mysecret
      securityContext:
        runAsUser: 5315
---
apiVersion: v1
kind: Service
metadata:
  name: videoingestion
spec:
  ports:
  - name: tcp
    port: 65012
    protocol: TCP
    targetPort: 65012
  selector:
    deployment-config.name: videoingestion
```

## 4-6. Web Visualizer(DeploymentConfig)

```bash
$ vi eii-deploy/dc-web.yaml 
```

```yaml:eii-deploy/dc-web.yaml
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: web
spec:
  replicas: 1
  selector:
    deployment-config.name: web
  template:
    metadata:
      labels:
        deployment-config.name: web
    spec:
      containers:
      - env:
        - name: AppName
          value: WebVisualizer
        - name: DEV_MODE
          value: "true"
        - name: PROFILING_MODE
          value: "false"
        - name: ETCD_HOST
          value: etcd
        - name: ETCD_CLIENT_PORT
          value: "8379"
        - name: ETCD_PREFIX
        - name: SUBSCRIBER_default_ENDPOINT
          value: videoanalytics:65013
        - name: SUBSCRIBER_default_TYPE
          value: zmq_tcp
        image: QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN/<Quayのユーザ名>/openedgeinsights/ia_web_visualizer:2.6.2
        imagePullPolicy: IfNotPresent
        name: ia-web-visualizer
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      imagePullSecrets:
      - name: quay-registry-mysecret
      securityContext:
        runAsUser: 5315
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  ports:
  - name: https
    port: 5000
    protocol: TCP
    targetPort: 5000
  - name: http
    port: 5001
    protocol: TCP
    targetPort: 5001
  selector:
    deployment-config.name: web
```

## 4-7. Route

```bash
$ vi eii-deploy/route.yaml 
```

```yaml:eii-deploy/route.yaml
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: eii-webvisualizer 
  namespace: eii
spec:
  path: /
  to:
    kind: Service
    name: web
  port:
    targetPort: 5001
```

# 5. Project eii、Service Account defaultへnonroot権限を付与

EIIは`UID：5315`で起動する様になっています。
デフォルトのままではOpenshiftのSCC([Security Context Constrains](https://rheb.hatenablog.com/entry/scc-process-on-openshift-202008#:~:text=Security%20Context%20Constraints(SCC)%E3%81%AF,Pod%E3%82%92%E8%B5%B7%E5%8B%95%E3%81%95%E3%81%9B%E3%81%BE%E3%81%99%E3%80%82))によって起動できません。(`openshift.io/sa.scc.uid-range=1000670000/10000`のため)

```
$ oc describe project eii
Name:			eii
Created:		19 hours ago
Labels:			kubernetes.io/metadata.name=eii
Annotations:		openshift.io/description=
			openshift.io/display-name=
			openshift.io/requester=system:admin
			openshift.io/sa.scc.mcs=s0:c26,c10
			openshift.io/sa.scc.supplemental-groups=1000670000/10000
			openshift.io/sa.scc.uid-range=1000670000/10000
Display Name:		<none>
Description:		<none>
Status:			Active
Node Selector:		<none>
Quota:			<none>
Resource limits:	<none>
```

そのため、Project`eii`にて、`runAsUser`で`UID`指定してPodを起動できる様に`nonroot`権限を付与します。

```
$ oc adm policy add-scc-to-user nonroot -z default -n eii
```

# 6. Quayレジストリの認証情報を設定するSecretを作成

```
$ oc create secret docker-registry quay-registry-mysecret --docker-server=QUAY_REGISTRY_NAME-PROJECT_NAME-quay.apps.CLUSTER_DOMEIN --docker-username=YOUR_USER_NAME --docker-password=YOUR_PASSWORD --docker-email=YOUR_MAIL_ADDRESS
```

# 7. Deloy

```bash
$ oc project eii

$ oc apply -f eii-provision/

$ oc get pods
NAME            READY   STATUS      RESTARTS   AGE
etcd-1-deploy   0/1     Completed   0          27m
etcd-1-hgffw    2/2     Running     0          20m


$ oc apply -f eii-deploy/

$ oc get pods
NAME                      READY   STATUS      RESTARTS   AGE
etcd-1-deploy             0/1     Completed   0          20h
etcd-1-hgffw              2/2     Running     0          20h
etcd-ui-1-7cfzh           1/1     Running     0          19h
etcd-ui-1-deploy          0/1     Completed   0          19h
videoanalytics-1-ccq6r    1/1     Running     0          19h
videoanalytics-1-deploy   0/1     Completed   0          19h
videoingestion-1-8wfsk    1/1     Running     0          19h
videoingestion-1-deploy   0/1     Completed   0          19h
web-1-deploy              0/1     Completed   0          19h
web-1-zvjt6               1/1     Running     0          19h

```

# 8. Routeの設定確認

```
$ oc get routes

NAME                HOST/PORT                                                        PATH   SERVICES   PORT   TERMINATION   WILDCARD
eii-webvisualizer   eii-webvisualizer-eii.apps.CLUSTER_DOMAINE   /      web        5001                 None

```

`https://eii-webvisualizer-eii.apps.CLUSTER_DOMAIN`へアクセスし、下記のデモ画面が表示されたら正常にdeployできてます。

![1639556143789.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/559968/23c78487-4063-8fbe-9d80-1b7b0ce6b820.png)