Dockerコンテナ上でHeapStatsをアタッチしたTomcatの起動

作業メモとして記載している為、不適切な内容が含まれている可能性がございます。

# 目標
1. VirtualBox上のRHEL7.5環境でDockerEngineをセットアップする
2. Tomcatのdocker-imageを起動させる
3. Dockerコンテナ内で動作するTomcatにHeapStatsをアタッチする
4. HeapStatsから出力されたログを永続化領域に出力させるようにする
5. 作成したイメージをDockerHubから他の環境でpullできるようにする
6. DockerfileからDocker imageの作成

# 前提環境
macOS HighSierra バージョン10.13.6
VirtualBox バージョン 5.2.22 r126460 (Qt5.6.3)
Red Hat Enterprise Linux Server release 7.5 (Maipo)


# 1. VirtualBox上のRHEL7.5環境でDockerEngineをセットアップする
* インストール済みのパッケージのアップデート

```console
[root@localhost ~]# yum update
```

* DockerEngineインストール用のリポジトリの追加

```console
[root@localhost ~]# vim /etc/yum.repos.d/docker.repo
[root@localhost ~]# 
[root@localhost ~]# cat /etc/yum.repos.d/docker.repo
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
[root@localhost ~]#
```

* DockerEngineのインストール

```console
[root@localhost ~]# yum install docker-engine
```

* docker起動 & 起動確認

```console
[root@localhost ~]# systemctl start docker
[root@localhost ~]# systemctl status docker 
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: active (running) since 土 2019-01-12 12:59:17 JST; 6s ago
     Docs: https://docs.docker.com
 Main PID: 6092 (dockerd)
    Tasks: 17
   Memory: 17.7M
   CGroup: /system.slice/docker.service
           ├─6092 /usr/bin/dockerd
           └─6096 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.soc...

 1月 12 12:59:13 localhost.localdomain systemd[1]: Starting Docker Application Container Engine...
 1月 12 12:59:14 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:14.019821683+09:0...6"
 1月 12 12:59:15 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:15.355072839+09:0...s"
 1月 12 12:59:15 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:15.358828767+09:0...."
 1月 12 12:59:16 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:16.352890408+09:0...s"
 1月 12 12:59:17 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:17.089879918+09:0...."
 1月 12 12:59:17 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:17.152254950+09:0...n"
 1月 12 12:59:17 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:17.152321385+09:0...ce
 1月 12 12:59:17 localhost.localdomain dockerd[6092]: time="2019-01-12T12:59:17.233900068+09:0...k"
 1月 12 12:59:17 localhost.localdomain systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost ~]# 
[root@localhost ~]# docker ps 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@localhost ~]# 
```

* テスト用docker-imageの起動

```console
[root@localhost ~]# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete 
Digest: sha256:2557e3c07ed1e38f26e389462d03ed943586f744621577a99efb77324b0fe535
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

[root@localhost ~]# 
```

参考にしたサイト
http://docs.docker.jp/v1.12/engine/installation/linux/rhel.html


# 2. Tomcatのdocker-imageを起動させる
* DockerHubからtomcatのdocker-imageを検索

```console
[root@localhost ~]# docker search tomcat
NAME                                       DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
tomcat                                     Apache Tomcat is an open source implementa...   2255      [OK]       
tomee                                      Apache TomEE is an all-Apache Java EE cert...   60        [OK]       
dordoka/tomcat                             Ubuntu 14.04, Oracle JDK 8 and Tomcat 8 ba...   51                   [OK]
davidcaste/alpine-tomcat                   Apache Tomcat 7/8 using Oracle Java 7/8 wi...   34                   [OK]
bitnami/tomcat                             Bitnami Tomcat Docker Image                     27                   [OK]
consol/tomcat-7.0                          Tomcat 7.0.57, 8080, "admin/admin"              16                   [OK]
cloudesire/tomcat                          Tomcat server, 6/7/8                            14                   [OK]
tutum/tomcat                               Base docker image to run a Tomcat applicat...   11                   
aallam/tomcat-mysql                        Debian, Oracle JDK, Tomcat & MySQL              10                   [OK]
meirwa/spring-boot-tomcat-mysql-app        a sample spring-boot app using tomcat and ...   10                   [OK]
jeanblanchard/tomcat                       Minimal Docker image with Apache Tomcat         8                    
arm32v7/tomcat                             Apache Tomcat is an open source implementa...   6                    
rightctrl/tomcat                           CentOS , Oracle Java, tomcat application s...   3                    [OK]
maluuba/tomcat7-java8                      Tomcat7 with java8.                             3                    
fabric8/tomcat-8                           Fabric8 Tomcat 8 Image                          2                    [OK]
arm64v8/tomcat                             Apache Tomcat is an open source implementa...   2                    
amd64/tomcat                               Apache Tomcat is an open source implementa...   2                    
99taxis/tomcat7                            Tomcat7                                         1                    [OK]
cfje/tomcat-resource                       Tomcat Concourse Resource                       0                    
1and1internet/debian-9-java-8-tomcat-8.5   Our tomcat 8.5 image                            0                    [OK]
jelastic/tomcat                            An image of the Tomcat Java application se...   0                    
swisstopo/service-print-tomcat             backend tomcat for service-print "the true...   0                    
oobsri/tomcat8                             Testing CI Jobs with different names.           0                    
picoded/tomcat7                            tomcat7 with jre8 and MANAGER_USER / MANAG...   0                    [OK]
s390x/tomcat                               Apache Tomcat is an open source implementa...   0                    
[root@localhost ~]#
```
*memo*
- OFFICIAL：[OK]　→　Tomcat公式のDockerイメージ
- STARS　→　Docker Hubのユーザがお気に入りとして指定した数（数が多い程人気が高い=信頼度が高い）

* tomcatイメージをローカル環境に取得

```console
[root@localhost ~]# docker pull tomcat
Using default tag: latest
latest: Pulling from library/tomcat
cd8eada9c7bb: Pull complete 
c2677faec825: Pull complete 
fcce419a96b1: Pull complete 
00d7fcb5828a: Pull complete 
f8c860563d60: Pull complete 
46f754145d12: Pull complete 
bff15a87739e: Pull complete 
6b5d3f0336b9: Pull complete 
1d858972966c: Pull complete 
4f2dc2e148d0: Pull complete 
db18a2f7d48d: Pull complete 
Digest: sha256:ff678df6c37e7837b80d7b4532ce49f7b152545a7bb8fa987701d13516129141
Status: Downloaded newer image for tomcat:latest
[root@localhost ~]# 
[root@localhost ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hello-world         latest              fce289e99eb9        11 days ago         1.84kB
tomcat              latest              1a51cb5e3006        13 days ago         462MB
[root@localhost ~]#
```

* tomcatイメージの起動

```console
[root@localhost ~]# docker run -d tomcat
2701c3f7cbb14f5e56836f8b596775790fa47926cbccd003d4d2786a5619743d
[root@localhost ~]#
[root@localhost ~]# docker ps 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
2701c3f7cbb1        tomcat              "catalina.sh run"   3 minutes ago       Up 3 minutes        8080/tcp            hardcore_roentgen
[root@localhost ~]# 
```
*memo* 
- docker run : dockerイメージからコンテナの作成&コンテナの起動（作成されたコンテナはstart/stopで起動/停止可能）
- -d : dockerコンテナをバックグラウンドで起動

* dockerコンテナの起動/停止

```console
[root@localhost ~]# docker ps -a 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                       PORTS               NAMES
2701c3f7cbb1        tomcat              "catalina.sh run"   7 minutes ago       Exited (143) 2 minutes ago                       hardcore_roentgen
faef0fde1c25        tomcat              "catalina.sh run"   10 minutes ago      Exited (130) 9 minutes ago                       flamboyant_jepsen
a9670ada1a09        hello-world         "/hello"            40 minutes ago      Exited (0) 40 minutes ago                        silly_euler
[root@localhost ~]# 
[root@localhost ~]# docker start faef0fde1c25
faef0fde1c25
[root@localhost ~]# 
[root@localhost ~]# docker stop faef0fde1c25
faef0fde1c25
[root@localhost ~]#
```

dockerコマンドリファレンス
http://docs.docker.jp/engine/reference/commandline/
dockerのライフサイクル
http://enakai00.hatenablog.com/entry/20140628/1403933390


# 3-1. Dockerコンテナ内で動作するTomcatにHeapStatsをアタッチする

`本来一般的には、DockerFileでコンテナ起動時にセットアップするものだと思いますが、一旦Dockerの知識習得の観点も踏まえ、イメージ内に全ての設定を含める方法で対応します`

* tomcatコンテナにログイン

```console
[root@localhost heapstats]# docker start faef0fde1c25
faef0fde1c25
[root@localhost heapstats]#
[root@localhost heapstats]# docker exec -it faef0fde1c25 /bin/bash
root@faef0fde1c25:/usr/local/tomcat#
```

* rpmやyumコマンドが打てない事が判明。。

```
root@faef0fde1c25:/usr/local/tomcat# rpm -qa | grep openjdk
bash: rpm: command not found
root@faef0fde1c25:/usr/local/tomcat# yum install openjdk
bash: yum: command not found
root@faef0fde1c25:/usr/local/tomcat# 
```
tomcatのイメージには、yum や rpmコマンドがインストールされていない模様。

rpm/yumコマンドが使える環境で以下を実行（rpmをtar形式で作成）

```console
[root@localhost rpm]# yum install yum-utils
[root@localhost rpm]# yum deplist rpm-4.11.3-35.el7.x86_64
```

* rpm-4.11.3-35.el7.x86_64に依存関係のあるrpmをダウンロード

```console
[root@localhost rpm]# cd /work/rpm/
[root@localhost rpm]# yumdownloader rpm-4.11.3-35.el7.x86_64 
[root@localhost rpm]# yumdownloader bash.x86_64 4.2.46-31.el7 
[root@localhost rpm]# yumdownloader libdb-utils.x86_64
[root@localhost rpm]# yumdownloader coreutils.x86_64
[root@localhost rpm]# yumdownloader curl.x86_64
[root@localhost rpm]# yumdownloader libacl.x86_64
[root@localhost rpm]# yumdownloader audit-libs.x86_64
[root@localhost rpm]# yumdownloader bzip2-libs.x86_64
[root@localhost rpm]# yumdownloader glibc.x86_64
[root@localhost rpm]# yumdownloader elfutils-libelf.x86_64
[root@localhost rpm]# yumdownloader lua.x86_64
[root@localhost rpm]# yumdownloader xz-libs.x86_64
[root@localhost rpm]# yumdownloader nss.x86_64
[root@localhost rpm]# yumdownloader popt.x86_64
[root@localhost rpm]# yumdownloader rpm-libs.x86_64
[root@localhost rpm]# yumdownloader libselinux.x86_64
[root@localhost rpm]# yumdownloader zlib.x86_64
[root@localhost rpm]# yumdownloader glibc.i686
```

rpmコマンドのファイル一式をtar形式に変換

```console
[root@localhost rpm]# mkdir rpm
[root@localhost rpm]# cd rpm
[root@localhost rpm]# rpm2cpio ../*.rpm | cpio -id
503 blocks
[root@localhost rpm]# ls -tlr 
合計 0
drwxr-xr-x. 2 root root  17  1月 12 16:23 bin
drwxr-xr-x. 5 root root  41  1月 12 16:23 usr
drwxr-xr-x. 3 root root  17  1月 12 16:23 var
drwxr-xr-x. 3 root root  38  1月 12 16:26 etc
drwxr-xr-x. 2 root root 102  1月 12 16:26 lib64
[root@localhost rpm]#
[root@localhost rpm]# tar zcvf ../rpm.tar.gz *
```

コンテナ内にファイルをコピー

```console
[root@localhost rpm]# docker cp rpm.tar.gz faef0fde1c25:/work/rpm/
```

コンテナ内でファイルの展開

```console
[root@localhost rpm]# docker exec -it faef0fde1c25 /bin/bash
root@faef0fde1c25:/# tar -zxvf /work/rpm/rpm.tar.gz
```

導入しようとしたrpmはRHELのrpmで作成したが、tomcatイメージのベースとなっているOSは、GNU/Linuxの模様。。。
一旦、Tomcatのコンテナイメージに直接heapstatsのrpm群を導入する方法は断念。

```
root@faef0fde1c25:/# rpm --version
rpm: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.1.2alpha' not found (required by /usr/lib64/librpmio.so.3)
root@faef0fde1c25:/# 
```
参考にしたサイト
https://polidog.jp/2011/09/27/centos6rpm/


# 3-2. CentOSをベースにTomcatのコンテナイメージを作成

* CentOSのコンテナイメージの起動

```console
[root@localhost rpm]# docker run centos
Unable to find image 'centos:latest' locally
latest: Pulling from library/centos
a02a4930cb5d: Pull complete 
Digest: sha256:184e5f35598e333bfa7de10d8fb1cebb5ee4df5bc0f970bf2b1e7c7345136426
Status: Downloaded newer image for centos:latest
[root@localhost rpm]#
[root@localhost rpm]# docker ps -a | grep centos
3ed7e1be949f        centos              "/bin/bash"              9 minutes ago       Exited (0) 20 seconds ago                         flamboyant_payne
[root@localhost rpm]#
```

* CentOSのイメージからコンテナの起動

```console
[root@localhost ~]# docker run -i -t -d centos /bin/bash
0e101e741e09a31721c593fc5222126711c880411aea8274f6e8a9ddbf1d9a79 
[root@localhost ~]# docker ps 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
0e101e741e09        centos              "/bin/bash"         16 seconds ago      Up 15 seconds                           happy_swanson
[root@localhost ~]# 
```

* tomcatのzipをCentOSのdockerイメージ内に転送

```console
[root@localhost work]# docker exec -it 0e101e741e09 /bin/bash
[root@0e101e741e09 /]# 
[root@0e101e741e09 /]# mkdir -p /work/tomcat
[root@0e101e741e09 /]# ls -l /work/
total 0
drwxr-xr-x. 2 root root 6 Jan 12 09:10 tomcat
[root@0e101e741e09 /]#
[root@0e101e741e09 /]# exit
exit
[root@0e101e741e09 /]#
[root@localhost work]# docker cp apache-tomcat-9.0.14.zip 0e101e741e09:/work/tomcat/
[root@localhost work]#  
```

* tomcatの導入&起動・停止

```console
[root@localhost work]# docker exec -it 0e101e741e09 /bin/bash
[root@0e101e741e09 /]# yum install unzip
[root@0e101e741e09 /]# yum install java
[root@0e101e741e09 /]# java -version 
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
[root@0e101e741e09 /]# 
[root@0e101e741e09 /]# cd /opt/
[root@0e101e741e09 opt]# unzip /work/tomcat/apache-tomcat-9.0.14.zip
[root@0e101e741e09 opt]# ls -l 
total 0
drwxr-xr-x. 9 root root 220 Dec  6 21:17 apache-tomcat-9.0.14
[root@0e101e741e09 opt]#
[root@0e101e741e09 opt]# ln -s apache-tomcat-9.0.14 tomcat
[root@0e101e741e09 opt]# ls -l 
total 0
drwxr-xr-x. 9 root root 220 Dec  6 21:17 apache-tomcat-9.0.14
lrwxrwxrwx. 1 root root  20 Jan 12 09:18 tomcat -> apache-tomcat-9.0.14
[root@0e101e741e09 opt]#
[root@0e101e741e09 opt]# chmod 755 -R /opt/tomcat/
[root@0e101e741e09 opt]# /opt/tomcat/bin/startup.sh  
Tomcat started.
[root@0e101e741e09 opt]# 
[root@0e101e741e09 opt]# ps -ef | grep java 
root       307     0 30 09:55 ?        00:00:08 /usr/bin/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start
root       348    27  0 09:55 ?        00:00:00 grep --color=auto java
[root@0e101e741e09 opt]#
[root@0e101e741e09 opt]# /opt/tomcat/bin/shutdown.sh 
[root@0e101e741e09 opt]#
```

※動作確認目的のため、一旦以下コマンドで実行権限を付与
chmod 755 -R /opt/tomcat/

* selinuxの無効化、firewalldの停止（これを実施しないと、仮想環境へのポートフォーワードでtomcatにアクセスできない。）

```console
[root@localhost ~]# vim /etc/selinux/config
>SELINUX=disabled
[root@localhost opt]# systemctl stop firewalld
[root@localhost opt]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@localhost opt]#
```
OS再起動の実施

※CentOSのイメージからコンテナ作成する際、ポートフォワードの設定を実施していなかったため、一旦tomcatを導入したcentosについて、Dockerイメージとして新規に作成する。

```console
[root@localhost ~]# docker ps -a 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                           PORTS               NAMES
0e101e741e09        centos              "/bin/bash"         23 hours ago        Exited (137) About an hour ago                       centos_tomcat
b94f9d43b69c        centos              "/bin/bash"         23 hours ago        Exited (137) 21 hours ago                            reverent_ride
3ed7e1be949f        centos              "/bin/bash"         23 hours ago        Exited (0) 23 hours ago                              flamboyant_payne
faef0fde1c25        tomcat              "catalina.sh run"   27 hours ago        Exited (143) 23 hours ago                            flamboyant_jepsen
[root@localhost ~]# 
[root@localhost ~]# docker ps 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@localhost ~]# 
[root@localhost ~]# docker commit -m "Add the Tomcat to the CentOS image" 0e101e741e09 centos_tomcat:latest
sha256:eab4fe739b5735ffea332f31ccc34a52e87704b2e02f4f17a96a946c8cbd33cd
[root@localhost ~]# 
[root@localhost ~]# docker images 
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
centos_tomcat       latest              eab4fe739b57        2 minutes ago       448MB
hello-world         latest              fce289e99eb9        12 days ago         1.84kB
tomcat              latest              1a51cb5e3006        2 weeks ago         462MB
centos              latest              1e1148e4cc2c        5 weeks ago         202MB
[root@localhost ~]#
```

* 作成したcentos_tomcatイメージから、ポートフォワードを指定しコンテナを起動

```console
[root@localhost ~]# docker run -it -d -p 28080:8080 --name centos_tomcat_portfw centos_tomcat
7ab6796ea8c2d2597df927ba71c27571c585cfd05114472fdde35a95b6582923
[root@localhost ~]#
```

* VirtualBoxの「ポートフォワーディング」の設定で、18080ポートへのアクセスを仮想環境の28080へフォワードするよう設定（仮想環境の28080へのアクセスをtomcatコンテナの8080にマッピング）
* 先ほど起動したコンテナにログインし、tomcatを起動

```console
[root@localhost ~]# docker exec -it centos_tomcat_portfw /bin/bash
[root@7ab6796ea8c2 /]# /opt/tomcat/bin/startup.sh 
Tomcat started.
[root@7ab6796ea8c2 /]#
```

以下リンクでtomcatのtopページが参照できる事を確認。
http://localhost:18080/

* HeapStats導入準備
 - 前提rpmの導入(openjdk-debuginfoはyumリポジトリに存在しないため個別にrpm導入)
 - heapstatsのrpmの導入

```console
[root@7ab6796ea8c2 logs]# yum install java-1.8.0-openjdk-devel
[root@7ab6796ea8c2 logs]# yum install java-1.8.0-openjdk-debug
[root@7ab6796ea8c2 work]# rpm -ivh /work/java-1.8.0-openjdk-debuginfo-1.8.0.191.b12-1.el7_6.x86_64.rpm
[root@7ab6796ea8c2 work]#
[root@7ab6796ea8c2 work]# rpm -qa | grep java 
python-javapackages-3.4.1-11.el7.noarch
tzdata-java-2018i-1.el7.noarch
java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64
java-1.8.0-openjdk-devel-1.8.0.191.b12-1.el7_6.x86_64
java-1.8.0-openjdk-debug-1.8.0.191.b12-1.el7_6.x86_64
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-headless-1.8.0.191.b12-1.el7_6.x86_64
java-1.8.0-openjdk-headless-debug-1.8.0.191.b12-1.el7_6.x86_64
java-1.8.0-openjdk-debuginfo-1.8.0.191.b12-1.el7_6.x86_64
[root@7ab6796ea8c2 work]#
```

heapstatsの最新版は以下からダウンロード
http://icedtea.wildebeest.org/download/heapstats/heapstats-2.1.0/bin/agent/el7/x86_64/

```console
[root@7ab6796ea8c2 work]# cd heapstats/
[root@7ab6796ea8c2 heapstats]# ls -tlr 
total 3260
-rw-r--r-- 1 root root 2546120 Jan 12 05:14 heapstats-debuginfo-2.1.0-0.el7.x86_64.rpm
-rw-r--r-- 1 root root  786704 Jan 12 05:14 heapstats-2.1.0-0.el7.x86_64.rpm
[root@7ab6796ea8c2 heapstats]#
[root@7ab6796ea8c2 heapstats]# yum localinstall ./*

```
* HeapStats動作確認

```console
[root@7ab6796ea8c2 heapstats]# java -agentlib:heapstats -version
heapstats INFO: HeapStats 2.1.0
heapstats INFO: Supported processor features: AVX
heapstats INFO: Agent Attach Enable = true
heapstats INFO: SnapShot FileName = heapstats_snapshot.dat
heapstats INFO: Heap Log FileName = heapstats_log.csv
heapstats INFO: Archive FileName = heapstats_analyze.zip
heapstats INFO: Console Log FileName = None (output to console)
heapstats INFO: LogLevel = INFO
heapstats INFO: ReduceSnapShot = true
heapstats INFO: CollectRefTree = true
heapstats INFO: Trigger on FullGC = true
heapstats INFO: Trigger on DumpRequest = true
heapstats INFO: Deadlock check = false
heapstats INFO: Log trigger on Error = true
heapstats INFO: Log trigger on Signal = true
heapstats INFO: Log trigger on Deadlock = true
heapstats INFO: RankingOrder = delta
heapstats INFO: RankLevel = 5
heapstats INFO: AlertPercentage = 50 ( 233144320 bytes )
heapstats INFO: Java heap usage alert percentage = 95 ( 422 MB )
heapstats INFO: Metaspace usage alert threshold 0 MB
heapstats INFO: SnapShot interval = 0 sec
heapstats INFO: Log interval = 300 sec
heapstats INFO: First collect log = true
heapstats INFO: Signal for normal logging is DISABLED.
heapstats INFO: Signal for all logging = SIGUSR2
heapstats INFO: Signal for config reloading = SIGHUP
heapstats INFO: Thread recorder = false
heapstats INFO: Buffer size of thread recorder = 100 MB
heapstats INFO: Thread record file name = heapstats-thread-records.htr
heapstats INFO: Thread record I/O tracer = /etc/heapstats/iotracer/IoTrace.class
heapstats INFO: Send SNMP Trap = false
heapstats INFO: SNMP target = localhost
heapstats INFO: SNMP community = public
heapstats INFO: NET-SNMP client library path = /usr/lib64/libnetsnmp.so
heapstats INFO: Temporary log directory = ./tmp
heapstats INFO: Archive command = "/usr/bin/zip %archivefile% -jr %logdir%"
heapstats INFO: Kill on Error = false
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
[root@7ab6796ea8c2 heapstats]#
```

* tomcat起動時にHeapStatsがアタッチされるようJVM起動オプションに設定

```console
[root@7ab6796ea8c2 bin]# vi /opt/tomcat/bin/setenv.sh
[root@7ab6796ea8c2 bin]# cat setenv.sh
CATALINA_OPTS="-agentlib:heapstats"
export CATALINA_OPTS
[root@7ab6796ea8c2 bin]#
```

* tomcat起動&HeapStats agentがアタッチされている事を確認&ログ出力確認

```console
[root@7ab6796ea8c2 logs]# ps -ef | grep java | grep -v grep 
root       497     0  0 09:19 ?        00:00:06 /usr/bin/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -agentlib:heapstats -Dignore.endorsed.dirs= -classpath /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start
[root@7ab6796ea8c2 logs]#
[root@7ab6796ea8c2 bin]# ls -ltr /opt/tomcat/bin/ | grep heapstats
-rw------- 1 root root    183 Jan 13 09:29 heapstats_log.csv
[root@7ab6796ea8c2 bin]# 
```

* HeapStatsのアタッチ設定が完了した断面でDockerイメージの作成

```console
[root@localhost ~]# docker ps -a 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                        PORTS               NAMES
7ab6796ea8c2        centos_tomcat       "/bin/bash"         2 hours ago         Exited (137) 39 seconds ago                       centos_tomcat_portfw
b94f9d43b69c        centos              "/bin/bash"         25 hours ago        Exited (137) 23 hours ago                         reverent_ride
3ed7e1be949f        centos              "/bin/bash"         25 hours ago        Exited (0) 25 hours ago                           flamboyant_payne
faef0fde1c25        tomcat              "catalina.sh run"   29 hours ago        Exited (143) 25 hours ago                         flamboyant_jepsen
[root@localhost ~]# 
[root@localhost ~]# 
[root@localhost ~]# docker commit -m "Add the HeapStats setting for Tomcat" 7ab6796ea8c2 tomcat_heapstats:latest
sha256:e78bcbc3ee3a8df646857a8ab6958493cdf54ecb18212dfb8afa1806fd261d8f
[root@localhost ~]# 
```

# 4. HeapStatsから出力されたログを永続化領域に出力させるようにする

* Dockerコンテナ内から同期する為の領域の作成

```console
[root@localhost ~]# mkdir -p /LOG/docker/
```

* 同期する領域を指定しイメージからコンテナの起動

```console
[root@localhost ~]# docker run -it -d -p 28080:8080 --name tomcat_heapstats_log -v /LOG/docker:/LOG tomcat_heapstats
5632f4174f49215f7c1dedf5d5e7e103deb98992cb382d6b9b0c3856b5b3df20
[root@localhost ~]# 
[root@localhost ~]# docker ps 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                     NAMES
5632f4174f49        tomcat_heapstats    "/bin/bash"         8 seconds ago       Up 7 seconds        0.0.0.0:28080->8080/tcp   tomcat_heapstats_log
[root@localhost ~]#
```
docker run -it -d -p 28080:8080 --name tomcat_heapstats_log -v ホストOSのpath(ここでは仮想環境のRHEL):コンテナ内のpath tomcat_heapstats

* テストログを出力し、フィルが共有されている事を確認

```console
[root@localhost ~]# docker exec -it tomcat_heapstats_log /bin/bash
[root@5632f4174f49 /]# cd LOG/
[root@5632f4174f49 LOG]# ls 
[root@5632f4174f49 LOG]# echo "share test" >> /LOG/test.log
[root@5632f4174f49 LOG]# 
[root@5632f4174f49 LOG]# ls -l /LOG
total 4
-rw-r--r-- 1 root root 11 Jan 13 09:50 test.log
[root@5632f4174f49 LOG]# 
[root@5632f4174f49 LOG]# exit
exit
[root@localhost ~]# cd /LOG
[root@localhost LOG]# ls 
docker
[root@localhost LOG]# cd docker/
[root@localhost docker]# ls 
test.log
[root@localhost docker]# 
```
ファイルが共有されている事を確認。

* 念のため、コンテナが削除されてもログファイルは永続化されている事を確認

```console
[root@localhost docker]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                     NAMES
5632f4174f49        tomcat_heapstats    "/bin/bash"         4 minutes ago       Up 4 minutes        0.0.0.0:28080->8080/tcp   tomcat_heapstats_log
[root@localhost docker]# 
[root@localhost docker]# docker stop 5632f4174f49
5632f4174f49
[root@localhost docker]# 
[root@localhost docker]# ls -trl 
合計 4
-rw-r--r-- 1 root root 11  1月 13 18:50 test.log
[root@localhost docker]# 
[root@localhost docker]# docker rm 5632f4174f49
5632f4174f49
[root@localhost docker]# 
[root@localhost docker]# docker ps -a 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                        PORTS               NAMES
7ab6796ea8c2        centos_tomcat       "/bin/bash"         2 hours ago         Exited (137) 17 minutes ago                       centos_tomcat_portfw
b94f9d43b69c        centos              "/bin/bash"         25 hours ago        Exited (137) 23 hours ago                         reverent_ride
3ed7e1be949f        centos              "/bin/bash"         25 hours ago        Exited (0) 25 hours ago                           flamboyant_payne
faef0fde1c25        tomcat              "catalina.sh run"   29 hours ago        Exited (143) 26 hours ago                         flamboyant_jepsen
[root@localhost docker]# 
[root@localhost docker]# ls -l 
合計 4
-rw-r--r-- 1 root root 11  1月 13 18:50 test.log
[root@localhost docker]# 
```

* 再度イメージからコンテナを起動し、ログイン後、heapstatsのログ出力先を同期ディレクトリ（/LOG）配下に指定

```console
[root@localhost docker]# docker exec -it tomcat_heapstats_log /bin/bash 
[root@944594a3101d /]# 
[root@944594a3101d /]# cd /etc/heapstats/                                 
[root@944594a3101d heapstats]# ls -tlr 
total 4
-rw-r--r-- 1 root root 1371 Mar 29  2018 heapstats.conf
drwxr-xr-x 2 root root   27 Jan 13 09:02 iotracer
[root@944594a3101d heapstats]# 
[root@944594a3101d heapstats]# 
[root@944594a3101d heapstats]# cp -p heapstats.conf heapstats.conf_20190113
[root@944594a3101d heapstats]# vi heapstats.conf
[root@944594a3101d heapstats]# 
[root@944594a3101d heapstats]# diff heapstats.conf heapstats.conf_20190113 
6,8c6,8
< file=/LOG/heapstats_snapshot.dat
< heaplogfile=/LOG/heapstats_log.csv
< archivefile=/LOG/heapstats_analyze.zip
---
> file=heapstats_snapshot.dat
> heaplogfile=heapstats_log.csv
> archivefile=heapstats_analyze.zip
44c44
< snapshot_interval=300
---
> snapshot_interval=0
[root@944594a3101d heapstats]# 
```

* tomcatの起動&ログ出力先確認

```console
[root@944594a3101d /]# /opt/tomcat/bin/startup.sh 
Tomcat started.
[root@944594a3101d /]# 
[root@944594a3101d /]# ls -tlr /LOG
total 276
-rw------- 1 root root     91 Jan 13 10:04 heapstats_log.csv
-rw------- 1 root root 274781 Jan 13 10:04 heapstats_snapshot.dat
[root@944594a3101d /]# 
[root@944594a3101d /]# exit
exit
[root@localhost /]# 
[root@localhost /]# ls -tlr /LOG/docker/
合計 276
-rw------- 1 root root     91  1月 13 19:04 heapstats_log.csv
-rw------- 1 root root 274781  1月 13 19:04 heapstats_snapshot.dat
[root@localhost /]#
```

今回はJVMのヒープやGC情報を収集する為のHeapStatsで確認したが、nmon等でも同様の考え方で対応可能。


# 5. 作成したイメージをDockerHubから他の環境でpullできるようにする

* DockerHubへのログイン

```console
[root@localhost docker]# docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: **********
Password: 
Login Succeeded
[root@localhost docker]#
```

* DockerHubへのpush前にtag付けが必要

```console
[root@localhost ~]# docker tag e78bcbc3ee3a 1429takahiro/tomcat_heapstats
```
`docker tag イメージID DockerHubのユーザ名/任意の名前`
`Dockerイメージ名のフォーマットは「リポジトリ名:タグ名」。タグ名を省略した場合は「latest」が自動で付与される`

* DockerHubへのpush

```console
[root@localhost ~]# docker push 1429takahiro/tomcat_heapstats
The push refers to a repository [docker.io/1429takahiro/tomcat_heapstats]
ed8c16f54fee: Pushed 
6780e5c38567: Pushed 
2f35c252b2e3: Pushed 
071d8bd76517: Pushed 
latest: digest: sha256:d20f75521a5a14cf9dc2ec4f055fbcbfc3a8a950d176a2be00c8b14df4dd5210 size: 1163
[root@localhost ~]# 
```

DockerHubにリポジトリとして登録されていることを確認
https://hub.docker.com/


# 6. DockerfileからDocker imageの作成

* Dockerfileの作成

```console
[root@localhost /]# cat /work/docker/Dockerfile 
# tomcatイメージから作成
FROM 1429takahiro/tomcat_heapstats

# Author
MAINTAINER 1429takahiro

# 公開port
EXPOSE 8080

# ディレクトリの作成
RUN mkdir /LOG

# ホストとボリュームの共有
VOLUME /LOG

# コンテナ起動時にtomcatの起動(フォアグラウンド)
CMD ["/opt/tomcat/bin/catalina.sh","run"]

[root@localhost /]#
```

* buildしDocker imageの作成

```console
[root@localhost docker]# docker build . -t 1429takahiro/tomcat_heapstats
Sending build context to Docker daemon  2.048kB
Step 1/6 : FROM 1429takahiro/tomcat_heapstats
 ---> 70962b616401
Step 2/6 : MAINTAINER 1429takahiro
 ---> Running in af998f501d6b
 ---> f092b14690a9
Removing intermediate container af998f501d6b
Step 3/6 : EXPOSE 8080
 ---> Running in 249264458d41
 ---> 6ad59f5f259f
Removing intermediate container 249264458d41
Step 4/6 : RUN mkdir /LOG
 ---> Running in 33fc1c437bb2
 ---> fbe53cc29806
Removing intermediate container 33fc1c437bb2
Step 5/6 : VOLUME /LOG
 ---> Running in 7369b0d34693
 ---> fea3d230eca0
Removing intermediate container 7369b0d34693
Step 6/6 : CMD /opt/tomcat/bin/catalina.sh run
 ---> Running in 96b2cdac1d80
 ---> 6306b54aebb9
Removing intermediate container 96b2cdac1d80
Successfully built 6306b54aebb9
Successfully tagged 1429takahiro/tomcat_heapstats:latest
[root@localhost docker]#
```
* DockerHubへpush

```console
[root@localhost docker]# docker push 1429takahiro/tomcat_heapstats:latest
The push refers to a repository [docker.io/1429takahiro/tomcat_heapstats]
91227a8c010a: Pushed 
6780e5c38567: Layer already exists 
2f35c252b2e3: Layer already exists 
071d8bd76517: Layer already exists 
latest: digest: sha256:5e19212c9cce0a023b884385daf8ee713591510f109a7139bdbfe06e5ec3ed13 size: 1161
[root@localhost docker]#
```

# 不明点

- portマッピングの設定（docker run "-p 28080:8080"）はDockerfileだけではできない？（必ずrun実行時に指定が必要？）
- Dockerfile内でVOLUMEを指定する事で、dockerコンテナ起動ホストの領域をマウントできるが、任意の場所を指定することはできない？（"-v /LOG/docker:/LOG"　のように指定したい。）