とにかくdocker registryを動かしたい

# 概要

CentOS7のdockerホストで、生のIPアドレスとオレオレ証明書でとにかくdocker registryを動かしたい人のための設定手順です。**本番で使う場合は適切な設定を入れてください。**

# 事前作業・前提条件

* 各ホストでdockerやdocker-composeが使える環境が構築されている
* opensslやそのほか必要なパッケージが入っている
* 作業ユーザは「vagrant」で各ホストにssh可能

| ホスト | IPアドレス | 役割 |
|:-----:|:---------:|:---------:|
| docker ホスト1 | 192.168.33.11 | docker registryコンテナが起動している |
| docker ホスト2 | 192.168.33.12| イメージをdocker registryにpushする |
| docker ホスト3 | 192.168.33.13| イメージをdocker registryからpullする |

# 作業手順

## dockerホスト1の設定
``` shell
[vagrant@192.168.33.11]
### IPアドレスで接続するための設定をv3_caセクション内に入れる
$ sudo vi /etc/pki/tls/openssl.cnf
================================================
...
[ v3_ca ]
...
# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy
subjectAltName = IP:192.168.33.11                       ### <-これを追加
================================================

### イメージとサーバ証明書を保存するディレクトリを作成
$ mkdir -p /home/vagrant/containers/regitstry /home/vagrant/containers/certs
$ cd /home/vagrant/containers/certs

### サーバ証明書の作成。 Common Name以外は良しなに入れてください。
$ openssl req -newkey rsa:4096 -nodes -sha256 -keyout domain.key -x509 -days 36500 -out domain.crt
================================================
Generating a 4096 bit RSA private key
.......++
.........................................................................++
writing new private key to 'domain.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:JP
State or Province Name (full name) []:Tokushima
Locality Name (eg, city) [Default City]:Tokushima-shi
Organization Name (eg, company) [Default Company Ltd]:ftakao2007 company
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:192.168.33.11    ### <- dockerホストのIPを入れる
Email Address []:
================================================
$ sudo chown root:root domain*

### dockerホスト2,3にサーバ証明書を配布する
$ sudo scp domain.crt vagrant@192.168.33.12:
$ sudo scp domain.crt vagrant@192.168.33.13:
```

## dockerホスト2,3でサーバ証明書を配置

``` shell
[vagrant@192.168.33.12, 192.168.33.13]
$ sudo mkdir -p /etc/docker/certs.d/192.168.33.11:5000
$ sudo chown root:root domain.crt
$ sudo mv domain.crt /etc/docker/certs.d/192.168.33.11:5000/ca.crt
```

## dockerホスト1でコンテナを立ち上げる

``` shell
[vagrant@192.168.33.11]
$ cd ~
$ vi docker-compose_registry.yml
================================================
registry:
  restart: always
  image: registry:2
  ports:
    - 5000:5000
  environment:
    REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt
    REGISTRY_HTTP_TLS_KEY: /certs/domain.key
    REGISTRY_STORAGE_DELETE_ENABLED: 'True'
  volumes:
    - /home/vagrant/containers/registry:/var/lib/registry
    - /home/vagrant/containers/certs:/certs
================================================
$ docker-compose -f docker-compose_registry.yml up -d
```

## dockerホスト2からイメージをpushする

``` shell
[vagrant@192.168.33.12]
### イメージIDの確認
$ docker images
================================================
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
centos              7                   0584b3d2cf6d        4 days ago          196.5 MB
================================================

### イメージにpushするためtagをつける
$ docker tag 0584b3d2cf6d 192.168.33.11:5000/centos:7
$ docker images
================================================
REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
192.168.33.11:5000/centos   7                   0584b3d2cf6d        4 days ago          196.5 MB
centos                      7                   0584b3d2cf6d        4 days ago          196.5 MB
================================================

### pushする
$ docker push 192.168.33.11:5000/centos:7
================================================
The push refers to a repository [192.168.33.11:5000/centos]
97ca462ad9ee: Pushed
7: digest: sha256:b2f9d1c0ff5f87a4743104d099a3d561002ac500db1b9bfa02a783a46e0d366c size: 529
================================================
```

## dockerホスト3でイメージをpullする

``` shell
[vagrant@192.168.33.13]
$ docker images
================================================
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
================================================

### イメージの確認
$ curl --insecure https://192.168.33.11:5000/v2/_catalog
================================================
{"repositories":["centos"]}
================================================

### タグの確認
$ curl --insecure https://192.168.33.11:5000/v2/centos/tags/list
================================================
{"name":"centos","tags":["7"]}
================================================

### イメージをpull
$ docker pull 192.168.33.11:5000/centos:7
================================================
7: Pulling from centos
08d48e6f1cff: Pull complete
Digest: sha256:b2f9d1c0ff5f87a4743104d099a3d561002ac500db1b9bfa02a783a46e0d366c
Status: Downloaded newer image for 192.168.33.11:5000/centos:7
================================================

$ docker images
================================================
REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
192.168.33.11:5000/centos   7                   0584b3d2cf6d        4 days ago          196.5 MB
================================================
```

# 参考

* [Deploying a registry server](https://docs.docker.com/registry/deploying/)
* [Failed tls handshake. Does not contain any IP SANs](http://serverfault.com/questions/611120/failed-tls-handshake-does-not-contain-any-ip-sans/611121)

<!--

# 削除方法(まだできてない)
``` shell
■ manifestsを消す

□ the Docker-Content-Digest is response.Header["Docker-Content-Digest"]

[jsd@localhost containers]$ curl --insecure -I https://10.136.3.187:5000/v2/centos/manifests/7 | grep Docker-Content-Digest
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  4231    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Docker-Content-Digest: sha256:52eb5a839619c0e67516f9be417f1fc45618e425c2372946eac2c27613d6e8c2  ### これ


delete v2/<repoName>/manifests/<Docker-Content-Digest>


================================================
■ blobを消す

□ the layerDigests is response.body["fsLayers"]["blobSum"]

[jsd@localhost ~]$ curl --insecure https://10.136.3.187:5000/v2/centos/manifests/7 | egrep 'fsLayers|blobSum' | sort -rn | uniq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4231  100  4231    0     0  53894      0 --:--:-- --:--:-- --:--:-- 53556
   "fsLayers": [
         "blobSum": "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"
         "blobSum": "sha256:08d48e6f1cff259389732d35307bb877215fa28867cdaff50c1dbd6e0b993c1f"


delete v2/<repoName>/blobs/<layerDigests>


■ CentOS7を消す例
[jsd@localhost ~]$ curl --insecure https://10.136.3.187:5000/v2/centos/tags/list
{"name":"centos","tags":["7"]}

[jsd@localhost ~]$ curl --insecure -I https://10.136.3.187:5000/v2/centos/manifests/7 | grep Docker-Content-Digest
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  4231    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Docker-Content-Digest: sha256:52eb5a839619c0e67516f9be417f1fc45618e425c2372946eac2c27613d6e8c2


[jsd@localhost ~]$ curl --insecure https://10.136.3.187:5000/v2/centos/manifests/7 | egrep 'fsLayers|blobSum' | sort -rn | uniq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4231  100  4231    0     0  53894      0 --:--:-- --:--:-- --:--:-- 53556
   "fsLayers": [
         "blobSum": "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"
         "blobSum": "sha256:08d48e6f1cff259389732d35307bb877215fa28867cdaff50c1dbd6e0b993c1f"







$ cd /home/vagrant/containers/registry/docker/registry/v2
$ find blobs > blobs.before
$ find repositories/centos > centos.before
$ curl -v -s -X DELETE "10.136.3.187:5000/v2/centos/manifests/sha256:52eb5a839619c0e67516f9be417f1fc45618e425c2372946eac2c27613d6e8c2"
================================================
* About to connect() to 10.136.3.187 port 5000 (#0)
*   Trying 10.136.3.187...
* Connected to 10.136.3.187 (10.136.3.187) port 5000 (#0)
> DELETE /v2/centos/manifests/sha256:52eb5a839619c0e67516f9be417f1fc45618e425c2372946eac2c27613d6e8c2 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.136.3.187:5000
> Accept: */*
>

* Connection #0 to host 10.136.3.187 left intact
================================================

$ curl -v -s -H "Accept: application/vnd.docker.distribution.manifest.v2+json" -X DELETE "10.136.3.187:5000/v2/centos/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"
================================================
* About to connect() to 10.136.3.187 port 5000 (#0)
*   Trying 10.136.3.187...
* Connected to 10.136.3.187 (10.136.3.187) port 5000 (#0)
> DELETE /v2/centos/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.136.3.187:5000
> Accept: */*
>

* Connection #0 to host 10.136.3.187 left intact
================================================

$ curl -v -s -X DELETE "10.136.3.187:5000/v2/centos/blobs/sha256:08d48e6f1cff259389732d35307bb877215fa28867cdaff50c1dbd6e0b993c1f"
================================================
* About to connect() to 10.136.3.187 port 5000 (#0)
*   Trying 10.136.3.187...
* Connected to 10.136.3.187 (10.136.3.187) port 5000 (#0)
> DELETE /v2/centos/blobs/sha256:08d48e6f1cff259389732d35307bb877215fa28867cdaff50c1dbd6e0b993c1f HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.136.3.187:5000
> Accept: */*
>

* Connection #0 to host 10.136.3.187 left intact
================================================


$ find blobs > blobs.after
$ find repositories/centos > centos.after
```

## 参考

* [Docker Registry 2.1 でプライベート レジストリを構築](https://academy.gmocloud.com/advance/20151005/256)
* [Delete images (repository + data) from registry #988](https://github.com/docker/docker-registry/issues/988)


-->