IBM Cloud: IBM ZにIBM Cloud Monitoring(Sysdig)を導入してみた

# 1. はじめに

IBM Cloudにはx86環境だけでなく、Power(UNIX)やIBM Z(ホスト）も提供されている。SysdigはIBM Z(Linux)環境でもサポートされているが、一般的にはx86環境でしか確認されていないように思えるので、IBM Z環境で試してみた。また、その際に、IBM Cloud docsで紹介されている方法で導入できるのか、何か別のworkaroundが必要なのかを確認してみた（結論としては、特別な考慮は必要なかった）。

なお、IBM Z環境下のSysdigは以下の制約がある。
https://docs.sysdig.com/en/docs/installation/sysdig-agent/agent-installation/agent-installation-requirements/#s390x-zlinux

また、IBM Zの仮想サーバーをプロビジョニングした際には、デフォルトで以下のようにモニタリング機能が稼働しており、（仮想サーバーにAgentを導入せずともハイパーバイザー経由のAgent less機能で）IBM Zの仮想サーバーのモニタリングができていることは確認できている。なお、こちらも内部的にはSysdigが使われている）
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/f9cb3248-1a29-b09b-a681-75f7331dcdad.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/d3ac3f0c-d4fe-26fc-8bc5-47480670f423.png)


# 2. 環境情報

```
root@z-syasuda:~# cat /etc/issue
Ubuntu 20.04.2 LTS \n \l

root@z-syasuda:~# uname -a
Linux z-syasuda 5.4.0-132-generic #148-Ubuntu SMP Mon Oct 17 16:01:44 UTC 2022 s390x s390x s390x GNU/Linux


root@z-syasuda:~# cat /proc/cpuinfo
vendor_id       : IBM/S390
# processors    : 1
bogomips per cpu: 1167.00
max thread id   : 0
features	: esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs vxe2 vxp sort dflt sie
facilities      : 0 1 2 3 4 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 30 31 32 33 34 35 36 37 38 40 41 42 43 44 45 47 48 49 50 51 52 53 54 57 58 59 60 61 64 65 69 71 73 74 75 76 77 78 80 81 82 129 130 131 133 134 135 138 139 146 147 148 150 151 152 155 156 161
cache0          : level=1 type=Data scope=Private size=128K line_size=256 associativity=8
cache1          : level=1 type=Instruction scope=Private size=128K line_size=256 associativity=8
cache2          : level=2 type=Data scope=Private size=4096K line_size=256 associativity=8
cache3          : level=2 type=Instruction scope=Private size=4096K line_size=256 associativity=8
cache4          : level=3 type=Unified scope=Shared size=262144K line_size=256 associativity=32
cache5          : level=4 type=Unified scope=Shared size=983040K line_size=256 associativity=60
processor 0: version = FF,  identification = 07C708,  machine = 8562

cpu number      : 0
cpu MHz dynamic : 4500
cpu MHz static  : 4500
root@z-syasuda:~#
root@z-syasuda:~#
root@z-syasuda:~#
root@z-syasuda:~# cat /proc/cpuinfo
vendor_id       : IBM/S390
# processors    : 1
bogomips per cpu: 1167.00
max thread id   : 0
features	: esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs vxe2 vxp sort dflt sie
facilities      : 0 1 2 3 4 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 30 31 32 33 34 35 36 37 38 40 41 42 43 44 45 47 48 49 50 51 52 53 54 57 58 59 60 61 64 65 69 71 73 74 75 76 77 78 80 81 82 129 130 131 133 134 135 138 139 146 147 148 150 151 152 155 156 161
cache0          : level=1 type=Data scope=Private size=128K line_size=256 associativity=8
cache1          : level=1 type=Instruction scope=Private size=128K line_size=256 associativity=8
cache2          : level=2 type=Data scope=Private size=4096K line_size=256 associativity=8
cache3          : level=2 type=Instruction scope=Private size=4096K line_size=256 associativity=8
cache4          : level=3 type=Unified scope=Shared size=262144K line_size=256 associativity=32
cache5          : level=4 type=Unified scope=Shared size=983040K line_size=256 associativity=60
processor 0: version = FF,  identification = 07C708,  machine = 8562

cpu number      : 0
cpu MHz dynamic : 4500
cpu MHz static  : 4500

root@z-syasuda:~# cat /proc/sysinfo
Manufacturer:         IBM
Type:                 8562
LIC Identifier:       305c25ff624c6845
Model:                A00              GT2
Sequence Code:        000000000004C708
Plant:                02
Model Capacity:       A00              00000000
Capacity Adj. Ind.:   100
Capacity Ch. Reason:  0
Capacity Transient:   0
Type 1 Percentage:    0
Type 2 Percentage:    0
Type 3 Percentage:    0
Type 4 Percentage:    0
Type 5 Percentage:    0

（途中略）

LPAR Number:          7
LPAR Characteristics: Shared
LPAR Name:            TOK1QZ07
LPAR Adjustment:      262
LPAR CPUs Total:      16
LPAR CPUs Configured: 16
LPAR CPUs Standby:    0
LPAR CPUs Reserved:   0
LPAR CPUs Dedicated:  0
LPAR CPUs Shared:     16
LPAR CPUs G-MTID:     0
LPAR CPUs S-MTID:     1
LPAR CPUs PS-MTID:    1
LPAR Extended Name:
LPAR UUID:            c5e679ee-2306-11ed-a82d-525412345600

VM00 Name:            k8s_039d
VM00 Control Program: KVM/Linux
VM00 Adjustment:      1000
VM00 CPUs Total:      1
VM00 CPUs Configured: 1
VM00 CPUs Standby:    0
VM00 CPUs Reserved:   0
VM00 Extended Name:   k8s_039dbe6794084c7cb514a276dd2345da_02e7_c12d11c7-b69f-4ac3-9d35-5e7f24eebc34
VM00 UUID:            c12d11c7-b69f-4ac3-9d35-5e7f24eebc34
```


# 3. Agentの導入および更新
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/e78e7932-6cbf-99e2-4b8e-f39a407d0e85.png)

以下のように本来は、Dockerで構成するべきらしいのだが、今回はAgentを直接導入してみた。更新も同様に実施すれば良い。

```
root@z-syasuda:~# curl -sL https://ibm.biz/install-sysdig-agent | sudo bash -s -- -a xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -c ingest.private.jp-tok.monitoring.cloud.ibm.com --collector_port 6443 --secure true -ac "sysdig_capture_enabled: false"
* Detecting operating system
------------
WARNING: A Docker container is the only officially supported platform on s390x
------------
* Installing gnupg
* Installing Sysdig public key
OK
* Installing Sysdig repository
* Installing kernel headers
* Installing Sysdig Agent
Extracting templates from packages: 100%
Preconfiguring packages ...
Selecting previously unselected package gcc-9-base:s390x.
(Reading database ... 54306 files and directories currently installed.)
Preparing to unpack .../gcc-9-base_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking gcc-9-base:s390x (9.4.0-1ubuntu1~20.04.1) ...
Preparing to unpack .../libc6_2.31-0ubuntu9.9_s390x.deb ...
Unpacking libc6:s390x (2.31-0ubuntu9.9) over (2.31-0ubuntu9.7) ...
Setting up libc6:s390x (2.31-0ubuntu9.9) ...
Selecting previously unselected package libisl22:s390x.
(Reading database ... 54311 files and directories currently installed.)
Preparing to unpack .../00-libisl22_0.22.1-1_s390x.deb ...
Unpacking libisl22:s390x (0.22.1-1) ...
Selecting previously unselected package libmpc3:s390x.
Preparing to unpack .../01-libmpc3_1.1.0-1_s390x.deb ...
Unpacking libmpc3:s390x (1.1.0-1) ...
Selecting previously unselected package cpp-9.
Preparing to unpack .../02-cpp-9_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking cpp-9 (9.4.0-1ubuntu1~20.04.1) ...
Selecting previously unselected package cpp.
Preparing to unpack .../03-cpp_4%3a9.3.0-1ubuntu2_s390x.deb ...
Unpacking cpp (4:9.3.0-1ubuntu2) ...
Selecting previously unselected package libcc1-0:s390x.
Preparing to unpack .../04-libcc1-0_10.3.0-1ubuntu1~20.04_s390x.deb ...
Unpacking libcc1-0:s390x (10.3.0-1ubuntu1~20.04) ...
Selecting previously unselected package binutils-common:s390x.
Preparing to unpack .../05-binutils-common_2.34-6ubuntu1.4_s390x.deb ...
Unpacking binutils-common:s390x (2.34-6ubuntu1.4) ...
Selecting previously unselected package libbinutils:s390x.
Preparing to unpack .../06-libbinutils_2.34-6ubuntu1.4_s390x.deb ...
Unpacking libbinutils:s390x (2.34-6ubuntu1.4) ...
Selecting previously unselected package libctf-nobfd0:s390x.
Preparing to unpack .../07-libctf-nobfd0_2.34-6ubuntu1.4_s390x.deb ...
Unpacking libctf-nobfd0:s390x (2.34-6ubuntu1.4) ...
Selecting previously unselected package libctf0:s390x.
Preparing to unpack .../08-libctf0_2.34-6ubuntu1.4_s390x.deb ...
Unpacking libctf0:s390x (2.34-6ubuntu1.4) ...
Selecting previously unselected package binutils-s390x-linux-gnu.
Preparing to unpack .../09-binutils-s390x-linux-gnu_2.34-6ubuntu1.4_s390x.deb ...
Unpacking binutils-s390x-linux-gnu (2.34-6ubuntu1.4) ...
Selecting previously unselected package binutils.
Preparing to unpack .../10-binutils_2.34-6ubuntu1.4_s390x.deb ...
Unpacking binutils (2.34-6ubuntu1.4) ...
Selecting previously unselected package libgomp1:s390x.
Preparing to unpack .../11-libgomp1_10.3.0-1ubuntu1~20.04_s390x.deb ...
Unpacking libgomp1:s390x (10.3.0-1ubuntu1~20.04) ...
Selecting previously unselected package libitm1:s390x.
Preparing to unpack .../12-libitm1_10.3.0-1ubuntu1~20.04_s390x.deb ...
Unpacking libitm1:s390x (10.3.0-1ubuntu1~20.04) ...
Selecting previously unselected package libatomic1:s390x.
Preparing to unpack .../13-libatomic1_10.3.0-1ubuntu1~20.04_s390x.deb ...
Unpacking libatomic1:s390x (10.3.0-1ubuntu1~20.04) ...
Selecting previously unselected package libasan5:s390x.
Preparing to unpack .../14-libasan5_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking libasan5:s390x (9.4.0-1ubuntu1~20.04.1) ...
Selecting previously unselected package libubsan1:s390x.
Preparing to unpack .../15-libubsan1_10.3.0-1ubuntu1~20.04_s390x.deb ...
Unpacking libubsan1:s390x (10.3.0-1ubuntu1~20.04) ...
Selecting previously unselected package libgcc-9-dev:s390x.
Preparing to unpack .../16-libgcc-9-dev_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking libgcc-9-dev:s390x (9.4.0-1ubuntu1~20.04.1) ...
Selecting previously unselected package gcc-9.
Preparing to unpack .../17-gcc-9_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking gcc-9 (9.4.0-1ubuntu1~20.04.1) ...
Selecting previously unselected package gcc.
Preparing to unpack .../18-gcc_4%3a9.3.0-1ubuntu2_s390x.deb ...
Unpacking gcc (4:9.3.0-1ubuntu2) ...
Selecting previously unselected package libdpkg-perl.
Preparing to unpack .../19-libdpkg-perl_1.19.7ubuntu3.2_all.deb ...
Unpacking libdpkg-perl (1.19.7ubuntu3.2) ...
Selecting previously unselected package make.
Preparing to unpack .../20-make_4.2.1-1.2_s390x.deb ...
Unpacking make (4.2.1-1.2) ...
Selecting previously unselected package dpkg-dev.
Preparing to unpack .../21-dpkg-dev_1.19.7ubuntu3.2_all.deb ...
Unpacking dpkg-dev (1.19.7ubuntu3.2) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../22-libc-dev-bin_2.31-0ubuntu9.9_s390x.deb ...
Unpacking libc-dev-bin (2.31-0ubuntu9.9) ...
Selecting previously unselected package linux-libc-dev:s390x.
Preparing to unpack .../23-linux-libc-dev_5.4.0-139.156_s390x.deb ...
Unpacking linux-libc-dev:s390x (5.4.0-139.156) ...
Selecting previously unselected package libcrypt-dev:s390x.
Preparing to unpack .../24-libcrypt-dev_1%3a4.4.10-10ubuntu4_s390x.deb ...
Unpacking libcrypt-dev:s390x (1:4.4.10-10ubuntu4) ...
Selecting previously unselected package libc6-dev:s390x.
Preparing to unpack .../25-libc6-dev_2.31-0ubuntu9.9_s390x.deb ...
Unpacking libc6-dev:s390x (2.31-0ubuntu9.9) ...
Selecting previously unselected package libstdc++-9-dev:s390x.
Preparing to unpack .../26-libstdc++-9-dev_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking libstdc++-9-dev:s390x (9.4.0-1ubuntu1~20.04.1) ...
Selecting previously unselected package g++-9.
Preparing to unpack .../27-g++-9_9.4.0-1ubuntu1~20.04.1_s390x.deb ...
Unpacking g++-9 (9.4.0-1ubuntu1~20.04.1) ...
Selecting previously unselected package g++.
Preparing to unpack .../28-g++_4%3a9.3.0-1ubuntu2_s390x.deb ...
Unpacking g++ (4:9.3.0-1ubuntu2) ...
Selecting previously unselected package build-essential.
Preparing to unpack .../29-build-essential_12.8ubuntu1.1_s390x.deb ...
Unpacking build-essential (12.8ubuntu1.1) ...
Selecting previously unselected package dctrl-tools.
Preparing to unpack .../30-dctrl-tools_2.24-3_s390x.deb ...
Unpacking dctrl-tools (2.24-3) ...
Selecting previously unselected package dkms.
Preparing to unpack .../31-dkms_2.8.1-5ubuntu2_all.deb ...
Unpacking dkms (2.8.1-5ubuntu2) ...
Selecting previously unselected package draios-agent.
Preparing to unpack .../32-draios-agent_12.11.0_s390x.deb ...
Unpacking draios-agent (12.11.0) ...
Selecting previously unselected package libfakeroot:s390x.
Preparing to unpack .../33-libfakeroot_1.24-1_s390x.deb ...
Unpacking libfakeroot:s390x (1.24-1) ...
Selecting previously unselected package fakeroot.
Preparing to unpack .../34-fakeroot_1.24-1_s390x.deb ...
Unpacking fakeroot (1.24-1) ...
Selecting previously unselected package libalgorithm-diff-perl.
Preparing to unpack .../35-libalgorithm-diff-perl_1.19.03-2_all.deb ...
Unpacking libalgorithm-diff-perl (1.19.03-2) ...
Selecting previously unselected package libalgorithm-diff-xs-perl.
Preparing to unpack .../36-libalgorithm-diff-xs-perl_0.04-6_s390x.deb ...
Unpacking libalgorithm-diff-xs-perl (0.04-6) ...
Selecting previously unselected package libalgorithm-merge-perl.
Preparing to unpack .../37-libalgorithm-merge-perl_0.08-3_all.deb ...
Unpacking libalgorithm-merge-perl (0.08-3) ...
Selecting previously unselected package libfile-fcntllock-perl.
Preparing to unpack .../38-libfile-fcntllock-perl_0.22-3build4_s390x.deb ...
Unpacking libfile-fcntllock-perl (0.22-3build4) ...
Selecting previously unselected package manpages-dev.
Preparing to unpack .../39-manpages-dev_5.05-1_all.deb ...
Unpacking manpages-dev (5.05-1) ...
Setting up manpages-dev (5.05-1) ...
Setting up libfile-fcntllock-perl (0.22-3build4) ...
Setting up libalgorithm-diff-perl (1.19.03-2) ...
Setting up binutils-common:s390x (2.34-6ubuntu1.4) ...
Setting up linux-libc-dev:s390x (5.4.0-139.156) ...
Setting up libctf-nobfd0:s390x (2.34-6ubuntu1.4) ...
Setting up libgomp1:s390x (10.3.0-1ubuntu1~20.04) ...
Setting up libfakeroot:s390x (1.24-1) ...
Setting up fakeroot (1.24-1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode
Setting up make (4.2.1-1.2) ...
Setting up libmpc3:s390x (1.1.0-1) ...
Setting up libatomic1:s390x (10.3.0-1ubuntu1~20.04) ...
Setting up libdpkg-perl (1.19.7ubuntu3.2) ...
Setting up libubsan1:s390x (10.3.0-1ubuntu1~20.04) ...
Setting up libcrypt-dev:s390x (1:4.4.10-10ubuntu4) ...
Setting up libisl22:s390x (0.22.1-1) ...
Setting up libbinutils:s390x (2.34-6ubuntu1.4) ...
Setting up libc-dev-bin (2.31-0ubuntu9.9) ...
Setting up libalgorithm-diff-xs-perl (0.04-6) ...
Setting up libcc1-0:s390x (10.3.0-1ubuntu1~20.04) ...
Setting up dctrl-tools (2.24-3) ...
Setting up libitm1:s390x (10.3.0-1ubuntu1~20.04) ...
Setting up gcc-9-base:s390x (9.4.0-1ubuntu1~20.04.1) ...
Setting up libalgorithm-merge-perl (0.08-3) ...
Setting up libctf0:s390x (2.34-6ubuntu1.4) ...
Setting up binutils-s390x-linux-gnu (2.34-6ubuntu1.4) ...
Setting up binutils (2.34-6ubuntu1.4) ...
Setting up dpkg-dev (1.19.7ubuntu3.2) ...
Setting up libasan5:s390x (9.4.0-1ubuntu1~20.04.1) ...
Setting up cpp-9 (9.4.0-1ubuntu1~20.04.1) ...
Setting up libc6-dev:s390x (2.31-0ubuntu9.9) ...
Setting up libgcc-9-dev:s390x (9.4.0-1ubuntu1~20.04.1) ...
Setting up cpp (4:9.3.0-1ubuntu2) ...
Setting up gcc-9 (9.4.0-1ubuntu1~20.04.1) ...
Setting up libstdc++-9-dev:s390x (9.4.0-1ubuntu1~20.04.1) ...
Setting up gcc (4:9.3.0-1ubuntu2) ...
Setting up dkms (2.8.1-5ubuntu2) ...
Setting up draios-agent (12.11.0) ...
Loading new draios-agent-12.11.0 DKMS files...
Building for 5.4.0-132-generic
Building initial module for 5.4.0-132-generic
Done.

sysdigcloud-probe.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.0-132-generic/updates/dkms/

depmod...

DKMS: install completed.
Setting up g++-9 (9.4.0-1ubuntu1~20.04.1) ...
Setting up g++ (4:9.3.0-1ubuntu2) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up build-essential (12.8ubuntu1.1) ...
Processing triggers for systemd (245.4-4ubuntu3.15) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.7) ...
* Setting access key
* Setting collector endpoint
* Setting collector port
* Setting connection security
* Adding additional configuration to dragent.yaml
```


# 4. 導入後のAgent情報確認

```
root@z-syasuda:~# /opt/draios/bin/dragent --version
12.11.0

root@z-syasuda:~# cat /opt/draios/etc/dragent.yaml
customerid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
collector: ingest.private.jp-tok.monitoring.cloud.ibm.com
collector_port: 6443
ssl: true
sysdig_capture_enabled: false

root@z-syasuda:~# systemctl status dragent
● dragent.service - LSB: Example initscript
     Loaded: loaded (/etc/init.d/dragent; generated)
     Active: active (running) since Wed 2023-02-15 22:58:26 UTC; 8min ago
       Docs: man:systemd-sysv-generator(8)
      Tasks: 40 (limit: 4778)
     Memory: 108.3M
     CGroup: /system.slice/dragent.service
             ├─9028 /opt/draios/bin/dragent --noipcns
             ├─9073 statsite -f /opt/draios/etc/statsite.ini
             ├─9074 /opt/draios/bin/dragent --noipcns
             ├─9075 /usr/bin/python3.8 /opt/draios/bin/sdchecks run
             ├─9076 /opt/draios/bin/dragent --noipcns
             ├─9077 cointerface --log_file /opt/draios/logs/k8s_klog.log --log_file_max_size 10 -v 0 --feature-gates K8sCommand=true
             ├─9078 promscrape_v2 --grpc.address=unix:/opt/draios/run/promscrape.sock --web.enable --web.listen-address=127.0.0.1:9990 --log.format=json --log.level=info --config.file=/opt/draios/etc/promscrape.yaml --source.label=pod_id,sys>
             └─9131 /opt/draios/bin/dragent --noipcns

Feb 15 22:58:26 z-syasuda systemd[1]: Starting LSB: Example initscript...
Feb 15 22:58:26 z-syasuda systemd[1]: Started LSB: Example initscript.
Feb 15 22:58:26 z-syasuda statsite[9036]: Starting statsite.
Feb 15 22:58:26 z-syasuda statsite[9036]: Listening on stdin.
Feb 15 22:58:26 z-syasuda statsite[9036]: Listening on tcp '127.0.0.1:8125'
Feb 15 22:58:26 z-syasuda statsite[9036]: Listening on udp '127.0.0.1:8125'.
Feb 15 22:58:38 z-syasuda statsite[9073]: Starting statsite.
Feb 15 22:58:38 z-syasuda statsite[9073]: Listening on stdin.
Feb 15 22:58:38 z-syasuda statsite[9073]: Listening on tcp '127.0.0.1:8125'
Feb 15 22:58:38 z-syasuda statsite[9073]: Listening on udp '127.0.0.1:8125'.



```


# 5. モニタリング画像の確認

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/0683d0f9-52a3-ed62-621c-280a48c50ac2.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/aa343caa-a2e4-bff9-591a-1caf5c5a2de9.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/62cbffc8-6b67-4a60-c25d-2c62f4cf8e53.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/9ec90677-dbe2-1342-3d53-f88e6adb4c68.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/4b403dae-8d2c-cf9c-8776-5ee0e7532134.png)