kube-aws を使ってお手軽に AWS 上に Kubernetes クラスタを構築

# 概要
CloudFormation を 使用した Kubernetes セットアップツール [kube-aws](https://github.com/coreos/coreos-kubernetes/tree/master/multi-node/aws) で AWS 上に Kubernetes 環境を構築する

特徴として以下の機能がある

- ELB integration for Kubernetes Services allows for traffic ingress to selected microservices
- Worker machines are deployed in an Auto Scaling group for effortless scaling
- Full TLS is set up between Kubernetes components and users interacting with kubectl

参考
https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html


# 環境
2016/01/14 時点で構築される内容は以下の通り

- CoreOS-alpha-891.0.0
- Docker version 1.9.1, build 4419fdb-dirty
- kubernetes v1.1.2

[VisualOps](https://www.visualops.io/) で VPC Import した際の構成図
![01.png](https://qiita-image-store.s3.amazonaws.com/0/105824/c9102164-8022-319a-e3bf-243c81cca476.png)

master node 1台、worker node 用 AutoScalingGroup が作成される

# 手順

**kube-aws を適当な場所にダウンロードして解凍**

```
$ wget https://github.com/coreos/coreos-kubernetes/releases/download/v0.3.0/kube-aws-linux-amd64.tar.gz
$ tar zxvf kube-aws-linux-amd64.tar.gz
```

**kube-aws の使い方**

```
$ ./kube-aws --help
Manage Kubernetes clusters on AWS

Usage:
  kube-aws [command]

Available Commands:
  destroy     Destroy an existing Kubernetes cluster
  render      Render a CloudFormation template
  status      Describe an existing Kubernetes cluster
  up          Create a new Kubernetes cluster
  version     Print version information and exit
  help        Help about any command

Flags:
      --aws-debug[=false]: Log debug information from aws-sdk-go library
      --config="cluster.yaml": Location of kube-aws cluster config file


Use "kube-aws [command] --help" for more information about a command.
```

**AWS Credentials を設定**

```
$ export AWS_ACCESS_KEY_ID="AKXXXXXXXXXXXXXXXXXX"
$ export AWS_SECRET_ACCESS_KEY="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
```

**サンプルの kubenetes クラスタ設定ファイルをダウンロード**

```
$ curl --silent --location https://raw.githubusercontent.com/coreos/coreos-kubernetes/master/multi-node/aws/cluster.yaml.example > cluster.yaml
```

**クラスタ設定**

```
$ vi cluster.yaml
```

```:cluster.yaml
# Unique name of Kubernetes cluster. In order to deploy
# more than one cluster into the same AWS account, this
# name must not conflict with an existing cluster.
clusterName: "kubernetes"

# Name of the SSH keypair already loaded into the AWS
# account being used to deploy this cluster.
keyName: "｛AWSに登録した Key pair name｝"

# Region to provision Kubernetes cluster
region: "ap-northeast-1"

# Availability Zone to provision Kubernetes cluster
availabilityZone: "ap-northeast-1a"

# DNS name routable to the Kubernetes controller nodes
# from worker nodes and external clients. The deployer
# is responsible for making this name routable
externalDNSName: "｛Kubernetes API にアクセスする際のドメイン名｝"

# Instance type for controller node
controllerInstanceType: "t2.micro"

# Disk size (GiB) for controller node
controllerRootVolumeSize: 10

# Number of worker nodes to create
workerCount: 3

# Instance type for worker nodes
workerInstanceType: "t2.micro"

# Disk size (GiB) for worker nodes
workerRootVolumeSize: 10

# Location of kube-aws artifacts used to deploy a new
# Kubernetes cluster. The necessary artifacts are already
# available in a public S3 bucket matching the version
# of the kube-aws tool. This parameter is typically
# overwritten only for development purposes.
#artifactURL: https://coreos-kubernetes.s3.amazonaws.com/<VERSION>

# CIDR for Kubernetes VPC
vpcCIDR: "10.0.0.0/16"

# CIDR for Kubernetes subnet
instanceCIDR: "10.0.0.0/24"

# IP Address for controller in Kubernetes subnet
controllerIP: 10.0.0.50

# CIDR for all service IP addresses
serviceCIDR: "10.3.0.0/24"

# CIDR for all pod IP addresses
podCIDR: "10.2.0.0/16"

# IP address of Kubernetes controller service (must be contained by serviceCIDR)
kubernetesServiceIP: "10.3.0.1"

# IP address of Kubernetes dns service (must be contained by serviceCIDR)
dnsServiceIP: "10.3.0.10"
```

4台でクラスタ構築
 - master :1台
 - worker node:3台 (AutoScalingGroup)

**デプロイ**

```
$ ./kube-aws up
Initialized TLS infrastructure
Wrote kubeconfig to /home/***/kube-aws/clusters/kubernetes/kubeconfig
Waiting for cluster creation...
Successfully created cluster

Cluster Name:   kubernetes
Controller IP:  xxx.xxx.xxx.xxx(EIP が表示される)
```

kube-aws によって作成される kubernetes 認証用の鍵と証明書は90日間しか使えないので注意。


*PRODUCTION NOTE*: the TLS keys and certificates generated by kube-aws should not be used to deploy a production Kubernetes cluster. Each component certificate is only valid for 90 days, while the CA is valid for 365 days. If deploying a production Kubernetes cluster, consider establishing PKI independently of this tool first.



**ドメインでアクセスできるようにする**

```
$ sudo vi /etc/hosts
```

以下を追記

```:/etc/hosts
｛Controller IP｝    ｛cluster.yaml に設定した externalDNSName｝
```

kubectl は作成された clusters/｛clusterName｝/kubeconfig を指定して実行する

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig cluster-info
Kubernetes master is running at https://｛externalDNSName｝
KubeDNS is running at https://｛externalDNSName｝/api/v1/proxy/namespaces/kube-system/services/kube-dns
```

node 確認

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get nodes
NAME                                            LABELS                                                                 STATUS    AGE
ip-10-0-0-171.ap-northeast-1.compute.internal   kubernetes.io/hostname=ip-10-0-0-171.ap-northeast-1.compute.internal   Ready     7m
ip-10-0-0-172.ap-northeast-1.compute.internal   kubernetes.io/hostname=ip-10-0-0-172.ap-northeast-1.compute.internal   Ready     7m
ip-10-0-0-173.ap-northeast-1.compute.internal   kubernetes.io/hostname=ip-10-0-0-173.ap-northeast-1.compute.internal   Ready     7m
```

以下コマンドで CloudFormation template が表示される

```
$ ./kube-aws render
```

```
{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Conditions": {
        "EmptyAvailabilityZone": {
            "Fn::Equals": [
                {
                    "Ref": "AvailabilityZone"
                },
                ""
            ]
        }
    },
    "Description": "kube-aws Kubernetes cluster",
    "Mappings": {
        "RegionMap": {
            "ap-northeast-1": {

...snip...
```

# Kubernetes で Wordpress を構築してみる

マニフェストは、[example](https://github.com/kubernetes/kubernetes/tree/release-1.1/examples/mysql-wordpress-pd) をベースに作成

**mysql データベース用の EBS Volume 作成**

```
$ aws ec2 create-volume --availability-zone ap-northeast-1a --size 10 --volume-type gp2
```

表示される VolumeId をメモっておく

**MySQL Pod 作成**

```:mysql.yaml
apiVersion: v1
kind: Pod
metadata:
  name: mysql
  labels:
    name: mysql
spec:
  containers:
    - resources:
        limits :
          cpu: 0.5
      image: mysql:5.6
      name: mysql
      env:
        - name: MYSQL_ROOT_PASSWORD
          value: yourpassword
      ports:
        - containerPort: 3306
          name: mysql
      volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
  volumes:
    - name: mysql-persistent-storage
      awsElasticBlockStore:
        volumeID: aws://ap-northeast-1a/｛上で作成した VolumeID｝
        fsType: ext4
```

Pod 作成

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f mysql.yaml
pod "mysql" created
```

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get pod
NAME        READY     STATUS    RESTARTS   AGE
mysql       1/1       Running   0          5m
```

**MySQL Service 作成**

```:mysql-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    name: mysql
  name: mysql
spec:
  ports:
    - port: 3306
  selector:
    name: mysql
```

Service 作成

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f mysql-service.yaml
service "mysql" created
```

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get svc
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)    SELECTOR         AGE
kubernetes   10.3.0.1     <none>        443/TCP    <none>           1h
mysql        10.3.0.170   <none>        3306/TCP   name=mysql       5m
```

**wordpress データ用の EBS Volume 作成**

```
$ aws ec2 create-volume --availability-zone ap-northeast-1a --size 10 --volume-type gp2
```

表示される VolumeId をメモっておく

**Wordpress Pod 作成**


```:wordpress.yaml
apiVersion: v1
kind: Pod
metadata:
  name: wordpress
  labels:
    name: wordpress
spec:
  containers:
    - image: wordpress
      name: wordpress
      env:
        - name: WORDPRESS_DB_PASSWORD
          value: yourpassword
      ports:
        - containerPort: 80
          name: wordpress
      volumeMounts:
        - name: wordpress-persistent-storage
          mountPath: /var/www/html
  volumes:
    - name: wordpress-persistent-storage
      awsElasticBlockStore:
        volumeID: aws://ap-northeast-1a/｛上で作成した VolumeID｝
        fsType: ext4
```

Pod 作成

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f wordpress.yaml
pod "mysql" wordpress
```

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get pod
NAME        READY     STATUS    RESTARTS   AGE
mysql       1/1       Running   0          8m
wordpress   1/1       Running   0          5m
```

**Wordpress Service 作成**

```:wordpress-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    name: wpfrontend
  name: wpfrontend
spec:
  ports:
    - port: 80
  selector:
    name: wordpress
  type: LoadBalancer
```

Service 作成

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f wordpress-service.yaml
service "mysql" created
```

```
$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get svc
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)    SELECTOR         AGE
kubernetes   10.3.0.1     <none>        443/TCP    <none>           55m
mysql        10.3.0.170   <none>        3306/TCP   name=mysql       9m
wpfrontend   10.3.0.130                 80/TCP     name=wordpress   5m
```

Service 作成の際に type: LoadBalancer を指定すると、自動で ELB が作成されるので確認

```
$ aws elb describe-load-balancers
{
    "LoadBalancerDescriptions": [
        {
            "Subnets": [
...snip...
```

表示された DNSName にブラウザでアクセス

![01.png](https://qiita-image-store.s3.amazonaws.com/0/105824/7162469e-74ea-8942-3263-7e7b54ef4285.png)


Wordpress のセットアップ画面が表示されたらOK


**掃除**

kubernetes で作成された LoadBalancer(ELB)、ELB 用 SecurityGroup は CloudFormation 管理外となり、削除されないので手動で実行

```
$ aws elb delete-load-balancer --load-balancer-name=｛LoadBalancerName｝
```

Volume も同様に削除

```
$ aws ec2 delete-volume --volume-id=｛VolumeID｝
```

destroy

```
$ ./kube-aws destroy
Destroyed cluster
```