minishift V1.28.0でnginx+gunicorn+Django2を動かす

# いままでのおさらい

Django2をminishift V1.28.0で動作させることには成功したわけですが…

- [Win10環境でDjango2のREST APIを試してみる](https://qiita.com/imp555sti/items/cbe579fbbfa3837075f3)
- [minishift V1.28.0でDjango2を動かす①：テンプレート確認](https://qiita.com/imp555sti/items/7fcb6d507c4e0eae5b3a)
- [minishift V1.28.0でDjango2を動かす②：テンプレート＆s2iソースの作成](https://qiita.com/imp555sti/items/728e774cd54e5781884b)
- [minishift V1.28.0でDjango2を動かす③：s2iソースの修正＆ビルド](https://qiita.com/imp555sti/items/f6ea232c1935f7c9042c)
- [minishift V1.28.0でDjango2を動かす④：残課題](https://qiita.com/imp555sti/items/ac7839d2a846f69a2bc1)
- [minishift V1.28.0でnginx+gunicorn+Django2を動かす](https://qiita.com/imp555sti/items/7733b48f41fc37ddda24)
- [minishift V1.28.0でnginx+gunicorn+Django2を動かす。あっcreatesuperuser忘れてた(笑)](https://qiita.com/imp555sti/items/e4ce4524fe8535313616)
- [openshiftに対応するためのDjango2コーディング時に気を付けるべき点](https://qiita.com/imp555sti/items/e9b37f0d5430220da690)


いくつかの課題が残っていました。

1. gunicornのバージョン番号がHTTPヘッダにもろ出し
2. 本番環境を見据えるとnginx+gunicornが良いらしい

という事で、minishift V1.28.0上でnginx+gunicorn+Django2を実現したいと思います。
まだ理解できていないところが多々あるので、ご指摘頂ければ幸いですm(_ _)m

# nginx+gunicorn+Django2をどう攻めるのか
minishiftつまりopenshift上でpython+nginx+gunicorn+Django2となると一筋縄ではいきません。
入念に作戦を練ります(^^♪

## containerの構成
考えに考えた末、以下のような構成で実現しようと思います。
1. `openshift/python:3.6`をベースに使いPython3.6+nginxをインストール、containerをImageStreamへ登録(`python3.6-nginx:latest`)
2. `python3.6-nginx:latest`をベースにs2iでgunicorn+Django2を実行

![図1.png](https://qiita-image-store.s3.amazonaws.com/0/324255/91ae3153-bded-7b74-8c79-66ed33138888.png)

gunicornおよびnginxの設定・チューニングは後からs2iビルドするソースで上書きできるよう設計しました。


## ベースとなるPython3.6+nginxのcontainerの作成

Git:
- [https://gitlab.com/imp555/python3.6-nginx.git](https://gitlab.com/imp555/python3.6-nginx.git)

Google先生で色々検索しながらサンプルとなるDockerfileを探します。
今回お手本にしたのはs2iベースのCentOS7にPython3.6を入れているようですので
それを参考にnginxをインストールさせるDockerfileを作成しますが…
結果的にたぶん、minishiftカタログ内にある`python:3.6`がベースになっている気がします(笑)

```docker:Dockerfile
# This image provides a Python 3.6 environment you can use to run your Python
# applications.
FROM centos/s2i-base-centos7

ENV PYTHON_VERSION=3.6 \
    PATH=$HOME/.local/bin/:$PATH \
    PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8 \
    LC_ALL=en_US.UTF-8 \
    LANG=en_US.UTF-8 \
    PIP_NO_CACHE_DIR=off

ENV SUMMARY="Platform for building and running Python $PYTHON_VERSION applications \
with nginx." \
    DESCRIPTION="Python $PYTHON_VERSION available as container is a base platform for \
building and running various Python $PYTHON_VERSION applications and frameworks with nginx."

LABEL summary="$SUMMARY" \
      description="$DESCRIPTION" \
      io.k8s.description="$DESCRIPTION" \
      io.k8s.display-name="Python $PYTHON_VERSION with nginx" \
      io.openshift.expose-services="8000:http" \
      io.openshift.tags="builder,python,python36,rh-python36,nginx" \
      com.redhat.component="python36-container" \
      name="python3.6-nginx" \
      version="1" \
      usage="s2i build https://gitlab.com/imp555/django2-nginx.git your_project_name/python3.6-nginx django2-nginx" \
      maintainer="imp555 <imp555@nethome.ne.jp>"

USER root

RUN INSTALL_PYTHON_PKGS="rh-python36 rh-python36-python-devel rh-python36-python-setuptools rh-python36-python-pip" && \
    INSTALL_NGINX_PKGS="nginx" && \
    yum install -y centos-release-scl && \
    yum install -y http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm && \
    yum -y --setopt=tsflags=nodocs install --enablerepo=centosplus $INSTALL_PYTHON_PKGS && \
    yum -y --setopt=tsflags=nodocs install --enablerepo=nginx $INSTALL_NGINX_PKGS && \
    rpm -V $INSTALL_PYTHON_PKGS $INSTALL_NGINX_PKGS && \
    yum -y clean all --enablerepo='*' && \
    ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime && \
    sed -i.bak 's/listen\(.*\)80;/listen 8000;/' /etc/nginx/conf.d/default.conf && \
    sed -i.bak 's/^user/#user/' /etc/nginx/nginx.conf && \
    nginx -t && \
    systemctl enable nginx

EXPOSE 8000

# Copy the S2I scripts from the specific language image to $STI_SCRIPTS_PATH.
COPY ./s2i/bin/ $STI_SCRIPTS_PATH

# Copy extra files to the image.
COPY ./root/ /


# - Create a Python virtual environment for use by any application to avoid
#   potential conflicts with Python packages preinstalled in the main Python
#   installation.
# - In order to drop the root user, we have to make some directories world
#   writable as OpenShift default security model is to run the container
#   under random UID.

RUN chown -R 1001:1001 /usr/share/nginx && \
    chown -R 1001:1001 /var/log/nginx && \
    touch /var/run/nginx.pid && \
    chown -R 1001:1001 /var/run/nginx.pid && \
    chmod 666 /var/run/nginx.pid && \
    chown -R 1001:1001 /etc/nginx && \
    source scl_source enable rh-python36 && \
    virtualenv ${APP_ROOT} && \
    chown -R 1001:0 ${APP_ROOT} && \
    chown -R 1001:0 ${STI_SCRIPTS_PATH} && \
    chmod -R +x ${STI_SCRIPTS_PATH} && \
    fix-permissions ${APP_ROOT} -P && \
    rpm-file-permissions && \
    ln -sf /dev/stdout /var/log/nginx/access.log && \
    ln -sf /dev/stderr /var/log/nginx/error.log

USER 1001

# Set the default CMD to print the usage of the language image.
CMD $STI_SCRIPTS_PATH/usage
```

- minishift(openshift)ではrootで実行されないため権限周りに苦労しました。
- nginxはTIME_ZONEを`Asia/Tokyo`に変更しています。
- nginxのログはそれぞれ`access.log`→`stdout`、`error.log`→`stderr`に出力されます。
- nginxのポートは`8000`としています。
- s2i用スクリプトのコピーをしていますが、ここでは要らないかもしれません(汗)
- 無駄なchmodやchownもあるかと思いますがそこはご愛敬(笑)
- usageに出力されるコマンドは確かめていません(笑)
- FROMのくだり、下のテンプレートで`openshift/python:3.6`に上書きされているはず(笑)
  本当は`FROM openshift/Python:3.6`かも

肝心のs2i Build時に実行されるスクリプトはこんな感じ
assembleについてはオリジナルのまま
runについてはnginxの起動を入れています。

```bash:s2i/bin/assemble
#!/bin/bash

function is_django_installed() {
  python -c "import django" &>/dev/null
}

function should_collectstatic() {
  is_django_installed && [[ -z "$DISABLE_COLLECTSTATIC" ]]
}

function virtualenv_bin() {
  if head "/etc/redhat-release" | grep -q "^Fedora"; then
    virtualenv-${PYTHON_VERSION} $1
  else
    virtualenv $1
  fi
}

# Install pipenv to the separate virtualenv to isolate it
# from system Python packages and packages in the main
# virtualenv. Executable is simlinked into ~/.local/bin
# to be accessible. This approach is inspired by pipsi
# (pip script installer).
function install_pipenv() {
  echo "---> Installing pipenv packaging tool ..."
  VENV_DIR=$HOME/.local/venvs/pipenv
  virtualenv_bin "$VENV_DIR"
  $VENV_DIR/bin/pip --isolated install -U pipenv
  mkdir -p $HOME/.local/bin
  ln -s $VENV_DIR/bin/pipenv $HOME/.local/bin/pipenv
}

set -e

shopt -s dotglob
echo "===> starting assemble script ..."
echo "---> Installing application source ..."
mv /tmp/src/* "$HOME"

if [[ -d $HOME/etc/nginx/ ]]; then
  echo "---> Installing nginx config ..."
  cp -rf $HOME/etc/nginx/* /etc/nginx/
  echo "---> Checking nginx config ..."
  /usr/sbin/nginx -t
fi

if [[ ! -z "$UPGRADE_PIP_TO_LATEST" ]]; then
  echo "---> Upgrading pip to latest version ..."
  pip install -U pip setuptools wheel
fi

if [[ ! -z "$ENABLE_PIPENV" ]]; then
  install_pipenv
  echo "---> Installing dependencies via pipenv ..."
  if [[ -f Pipfile ]]; then
    pipenv install --deploy
  elif [[ -f requirements.txt ]]; then
    pipenv install -r requirements.txt
  fi
  # pipenv check
elif [[ -f requirements.txt ]]; then
  echo "---> Installing dependencies ..."
  pip install -r requirements.txt
fi

if [[ -f setup.py && -z "$DISABLE_SETUP_PY_PROCESSING" ]]; then
  echo "---> Installing application ..."
  pip install .
fi

if should_collectstatic; then
  (
    echo "---> Collecting Django static files ..."

    APP_HOME=$(readlink -f "${APP_HOME:-.}")
    # Change the working directory to APP_HOME
    PYTHONPATH="$(pwd)${PYTHONPATH:+:$PYTHONPATH}"
    cd "$APP_HOME"

    # Look for 'manage.py' in the current directory
    manage_file=./manage.py

    if [[ ! -f "$manage_file" ]]; then
      echo "WARNING: seems that you're using Django, but we could not find a 'manage.py' file."
      echo "'manage.py collectstatic' ignored."
      exit
    fi

    if ! python $manage_file collectstatic --dry-run --noinput &> /dev/null; then
      echo "WARNING: could not run 'manage.py collectstatic'. To debug, run:"
      echo "    $ python $manage_file collectstatic --noinput"
      echo "Ignore this warning if you're not serving static files with Django."
      exit
    fi

    python $manage_file collectstatic --noinput
  )
fi

# set permissions for any installed artifacts
fix-permissions /opt/app-root
```

```bash:s2i/bin/run
#!/bin/bash
source /opt/app-root/etc/generate_container_user

set -e

function is_gunicorn_installed() {
  hash gunicorn &>/dev/null
}

function is_django_installed() {
  python -c "import django" &>/dev/null
}

function should_migrate() {
  is_django_installed && [[ -z "$DISABLE_MIGRATE" ]]
}

# Guess the number of workers according to the number of cores
function get_default_web_concurrency() {
  limit_vars=$(cgroup-limits)
  local $limit_vars
  if [ -z "${NUMBER_OF_CORES:-}" ]; then
    echo 1
    return
  fi

  local max=$((NUMBER_OF_CORES*2))
  # Require at least 43 MiB and additional 40 MiB for every worker
  local default=$(((${MEMORY_LIMIT_IN_BYTES:-MAX_MEMORY_LIMIT_IN_BYTES}/1024/1024 - 43) / 40))
  default=$((default > max ? max : default))
  default=$((default < 1 ? 1 : default))
  # According to http://docs.gunicorn.org/en/stable/design.html#how-many-workers,
  # 12 workers should be enough to handle hundreds or thousands requests per second
  default=$((default > 12 ? 12 : default))
  echo $default
}

function exec_nginx() {
  echo "---> Running nginx front-end server ..."
  /usr/sbin/nginx
}

echo "===> starting run script ..."

APP_HOME=$(readlink -f "${APP_HOME:-.}")
# Change the working directory to APP_HOME
PYTHONPATH="$(pwd)${PYTHONPATH:+:$PYTHONPATH}"
cd "$APP_HOME"

app_script_check="${APP_SCRIPT-}"
APP_SCRIPT="${APP_SCRIPT-app.sh}"
if [[ -f "$APP_SCRIPT" ]]; then
  echo "---> Running application from script ($APP_SCRIPT) ..."
  if [[ "$APP_SCRIPT" != /* ]]; then
    APP_SCRIPT="./$APP_SCRIPT"
  fi
  exec "$APP_SCRIPT"
else
  test -n "$app_script_check" && (>&2 echo "ERROR: file '$app_script_check' not found.") && exit 1
fi

app_file_check="${APP_FILE-}"
APP_FILE="${APP_FILE-app.py}"
if [[ -f "$APP_FILE" ]]; then
  echo "---> Running application from Python script ($APP_FILE) ..."
  exec python "$APP_FILE"
else
  test -n "$app_file_check" && (>&2 echo "ERROR: file '$app_file_check' not found.") && exit 1
fi

# Look for 'manage.py' in the current directory
manage_file=./manage.py

if should_migrate; then
  if [[ -f "$manage_file" ]]; then
    echo "---> Migrating database ..."
    python "$manage_file" migrate --noinput
  else
    echo "WARNING: seems that you're using Django, but we could not find a 'manage.py' file."
    echo "Skipped 'python manage.py migrate'."
  fi
fi

if is_gunicorn_installed; then
  setup_py=$(find "$HOME" -maxdepth 2 -type f -name 'setup.py' -print -quit)
  # Look for wsgi module in the current directory
  if [[ -z "$APP_MODULE" && -f "./wsgi.py" ]]; then
    APP_MODULE=wsgi
  elif [[ -z "$APP_MODULE" && -f "$setup_py" ]]; then
    APP_MODULE="$(python "$setup_py" --name)"
  fi

  if [[ "$APP_MODULE" ]]; then
    export WEB_CONCURRENCY=${WEB_CONCURRENCY:-$(get_default_web_concurrency)}

    exec_nginx
    echo "---> Serving application with gunicorn ($APP_MODULE) ..."
    gunicorn "$APP_MODULE" --bind=0.0.0.0:8080 --access-logfile=- --config "$APP_CONFIG"
  fi
fi

if is_django_installed; then
  if [[ -f "$manage_file" ]]; then
    echo "---> Serving application with 'manage.py runserver' ..."
    echo "WARNING: this is NOT a recommended way to run you application in production!"
    echo "Consider using gunicorn or some other production web server."
    exec_nginx
    python "$manage_file" runserver 0.0.0.0:8080
  else
    echo "WARNING: seems that you're using Django, but we could not find a 'manage.py' file."
    echo "Skipped 'python manage.py runserver'."
  fi
fi

>&2 echo "ERROR: don't know how to run your application."
>&2 echo "Please set either APP_MODULE, APP_FILE or APP_SCRIPT environment variables, or create a file 'app.py' to launch your application."
exit 1
```

上のDocker Imageを登録するためのyamlも作成しました。
minishift(openshift)のWeb-Consoleで`Import YAML/JSON`から自分のプロジェクトに`python3.6-nginx:latest`を登録できます。

```yaml:openshift/templates/python3.6-nginx.yaml
apiVersion: v1
kind: List
items:
- apiVersion: v1
  kind: ImageStream
  metadata:
    name: python3.6-nginx
- apiVersion: v1
  kind: BuildConfig
  metadata:
    name: python3.6-nginx
  spec:
    output:
      to:
        kind: ImageStreamTag
        name: python3.6-nginx:latest
    postCommit: {}
    resources: {}
    runPolicy: Serial
    source:
      git:
        uri: https://gitlab.com/imp555/python3.6-nginx.git
        ref: master
      type: Git
    strategy:
      dockerStrategy:
        forcePull: true
        from:
          kind: ImageStreamTag
          name: python:3.6
          namespace: openshift
      type: Docker
    triggers:
    - type: ConfigChange
    - imageChange: {}
      type: ImageChange
```

## gunicorn+Django2を入れたcontainerの作成
s2iでgunicornとDjango2プロジェクトを導入し、nginxとgunicornを起動します。

Git:
- [https://gitlab.com/imp555/django2-nginx.git](https://gitlab.com/imp555/django2-nginx.git)

サンプルDjango2 Projectを動作させるためのテンプレートファイルです。
minishift(openshift)のWeb-Consoleで`Import YAML/JSON`から自分のプロジェクトに`django2-nginx:latest`を登録できます。
その際、先に上の`python3.6-nginx:latest`を登録している必要があります。
また、テンプレート読み込み時に聞かれる`Namespace`については、自分のProject名を登録します。

```json:openshift/templates/django2-nginx.json
{
  "kind": "Template",
  "apiVersion": "v1",
  "metadata": {
    "name": "django2-nginx-ex",
    "annotations": {
      "openshift.io/display-name": "Django",
      "description": "An example Django application with nginx, no database. For more information about using this template, including OpenShift considerations, see https://gitlab.com/imp555/django2-nginx/master/README.md.",
      "tags": "quickstart,python,django,nginx",
      "iconClass": "icon-python",
      "openshift.io/long-description": "This template defines resources needed to develop a Django based application, including a build configuration and application deployment configuration.  It does not include a database.",
      "openshift.io/provider-display-name": "imp555",
      "openshift.io/documentation-url": "https://gitlab.com/imp555/django2-nginx",
      "template.openshift.io/bindable": "false"
    }
  },
  "labels": {
      "template": "django2-nginx-ex",
      "app": "django2-nginx-ex"
  },
  "message": "The following service(s) have been created in your project: ${NAME}.\n\nFor more information about using this template, including OpenShift considerations, see https://github.com/sclorg/django-ex/blob/master/README.md.",
  "objects": [
    {
      "kind": "Secret",
      "apiVersion": "v1",
      "metadata": {
        "name": "${NAME}"
      },
      "stringData" : {
        "django-secret-key" : "${DJANGO_SECRET_KEY}"
      }
    },
    {
      "kind": "Service",
      "apiVersion": "v1",
      "metadata": {
        "name": "${NAME}",
        "annotations": {
          "description": "Exposes and load balances the application pods"
        }
      },
      "spec": {
        "ports": [
          {
            "name": "http",
            "port": 8000,
            "targetPort": 8000
          }
        ],
        "selector": {
          "name": "${NAME}"
        }
      }
    },
    {
      "kind": "Route",
      "apiVersion": "v1",
      "metadata": {
        "name": "${NAME}"
      },
      "spec": {
        "host": "${APPLICATION_DOMAIN}",
        "to": {
          "kind": "Service",
          "name": "${NAME}"
        }
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "${NAME}",
        "annotations": {
          "description": "Keeps track of changes in the application image"
        }
      }
    },
    {
      "kind": "BuildConfig",
      "apiVersion": "v1",
      "metadata": {
        "name": "${NAME}",
        "annotations": {
          "description": "Defines how to build the application",
          "template.alpha.openshift.io/wait-for-ready": "true"
        }
      },
      "spec": {
        "source": {
          "type": "Git",
          "git": {
            "uri": "${SOURCE_REPOSITORY_URL}",
            "ref": "${SOURCE_REPOSITORY_REF}"
          }
        },
        "strategy": {
          "type": "Source",
          "sourceStrategy": {
            "forcePull": true,
            "from": {
              "kind": "ImageStreamTag",
              "namespace": "${NAMESPACE}",
              "name": "python${PYTHON_VERSION}-nginx:latest"
            },
            "env": [
              {
                  "name": "PIP_INDEX_URL",
                  "value": "${PIP_INDEX_URL}"
              }
            ]
          }
        },
        "output": {
          "to": {
            "kind": "ImageStreamTag",
            "name": "${NAME}:latest"
          }
        },
        "triggers": [
          {
            "type": "ImageChange"
          },
          {
            "type": "ConfigChange"
          },
          {
            "type": "GitHub",
            "github": {
              "secret": "${GITHUB_WEBHOOK_SECRET}"
            }
          }
       ],
        "postCommit": {
          "script": "python ./manage.py test"
        }
      }
    },
    {
      "kind": "DeploymentConfig",
      "apiVersion": "v1",
      "metadata": {
        "name": "${NAME}",
        "annotations": {
          "description": "Defines how to deploy the application server",
          "template.alpha.openshift.io/wait-for-ready": "true"
        }
      },
      "spec": {
        "strategy": {
          "type": "Rolling"
        },
        "triggers": [
          {
            "type": "ImageChange",
            "imageChangeParams": {
              "automatic": true,
              "containerNames": [
                "django2-nginx-ex"
              ],
              "from": {
                "kind": "ImageStreamTag",
                "name": "${NAME}:latest"
              }
            }
          },
          {
            "type": "ConfigChange"
          }
        ],
        "replicas": 1,
        "selector": {
          "name": "${NAME}"
        },
        "template": {
          "metadata": {
            "name": "${NAME}",
            "labels": {
              "name": "${NAME}"
            }
          },
          "spec": {
            "containers": [
              {
                "name": "${NAME}",
                "image": "${NAME}:latest",
                "ports": [
                  {
                    "containerPort": 8000,
                    "protocol": "TCP"
                  }
                ],
                "env": [
                  {
                    "name": "APP_CONFIG",
                    "value": "${APP_CONFIG}"
                  },
                  {
                    "name": "DJANGO_SECRET_KEY",
                    "valueFrom": {
                      "secretKeyRef" : {
                        "name" : "${NAME}",
                        "key" : "django-secret-key"
                      }
                    }
                  }
                ],
                "resources": {
                  "limits": {
                    "memory": "${MEMORY_LIMIT}"
                  }
                },
                "readinessProbe": {
                  "failuerThreshold": 3,
                  "successThreshold": 1,
                  "initialDelaySeconds": 30,
                  "periodSeconds": 10,
                  "timeoutSeconds": 3,
                  "tcpSocket": {
                    "port": 8000
                  }
                },
                "livenessProbe": {
                  "failuerThreshold": 3,
                  "successThreshold": 1,
                  "initialDelaySeconds": 30,
                  "periodSeconds": 10,
                  "timeoutSeconds": 3,
                  "httpGet": {
                    "path": "/",
                    "port": 8000
                  }
                }
              }
            ]
          }
        }
      }
    }
  ],
  "parameters": [
    {
      "name": "NAME",
      "displayName": "Name",
      "description": "The name assigned to all of the frontend objects defined in this template.",
      "required": true,
      "value": "django2-nginx-ex"
    },
    {
      "name": "NAMESPACE",
      "displayName": "Namespace",
      "required": true,
      "description": "The OpenShift Namespace where the ImageStream resides.",
      "value": "openshift"
    },
    {
      "name": "PYTHON_VERSION",
      "displayName": "Version of Python Image",
      "description": "Version of Python image to be used (3.6 or latest).",
      "value": "3.6",
      "required": true
    },
    {
      "name": "MEMORY_LIMIT",
      "displayName": "Memory Limit",
      "required": true,
      "description": "Maximum amount of memory the container can use.",
      "value": "256Mi"
    },
    {
      "name": "SOURCE_REPOSITORY_URL",
      "displayName": "Git Repository URL",
      "required": true,
      "description": "The URL of the repository with your application source code.",
      "value": "https://gitlab.com/imp555/django2-nginx.git"
    },
    {
      "name": "SOURCE_REPOSITORY_REF",
      "displayName": "Git Reference",
      "description": "Set this to a branch name, tag or other ref of your repository if you are not using the default branch.",
      "value": "master"
    },
    {
      "name": "CONTEXT_DIR",
      "displayName": "Context Directory",
      "description": "Set this to the relative path to your project if it is not in the root of your repository."
    },
    {
      "name": "APPLICATION_DOMAIN",
      "displayName": "Application Hostname",
      "description": "The exposed hostname that will route to the Django service, if left blank a value will be defaulted.",
      "value": ""
    },
    {
      "name": "GITHUB_WEBHOOK_SECRET",
      "displayName": "GitHub Webhook Secret",
      "description": "Github trigger secret.  A difficult to guess string encoded as part of the webhook URL.  Not encrypted.",
      "generate": "expression",
      "from": "[a-zA-Z0-9]{40}"
    },
    {
      "name": "APP_CONFIG",
      "displayName": "Application Configuration File Path",
      "description": "Relative path to Gunicorn configuration file (optional).",
      "value": "conf/gunicorn.py"
    },
    {
      "name": "DJANGO_SECRET_KEY",
      "displayName": "Django Secret Key",
      "description": "Set this to a long random string.",
      "generate": "expression",
      "from": "[\\w]{50}"
    },
    {
      "name": "PIP_INDEX_URL",
      "displayName": "Custom PyPi Index URL",
      "description": "The custom PyPi index URL",
      "value": ""
    }
  ]
}
```

- Namespaceは自分のProjectを登録してください。(つまりDockerfileで`python3.6-nginx:latest`をインポートしたNamespaceです。)
- nginxのアクセスログとgunicornのアクセスログが重複して`stdout`に出力されるため、nginxのアクセスログをoffにしています。
- nginxは`8000`番ポート、gunicornは`8080`番ポートで待ち受けを行っており、Pod外部へは`8000`番ポートを開放しています。
- `readinessProbe`は8000番ポートに対しTCPで通信が行えるかを確認しています。
- `livenessProbe`は8000番ポートにGETでアクセスし200_OKが返ることを確認しています。

# 実際にやってみる(web-console編)

## プロジェクト(`django2-nginx`)を作成する
まずはProjectを作成します。仮に`django2-nginx`としました。
![60.png](https://qiita-image-store.s3.amazonaws.com/0/324255/fc3a61f1-61f0-0c10-63db-668a19294661.png)

## ベースとなる`python3.6-nginx:latest`をビルドする
Overview画面で`Import YAML/JSON`をクリックします。
![61.png](https://qiita-image-store.s3.amazonaws.com/0/324255/a22acb92-7b6b-0b41-3868-193c04168696.png)
Gitにある`openshift/templates/python3.6-nginx.yaml`をRAWでCOPYし貼り付け、Createボタンをクリックします。
Git: [https://gitlab.com/imp555/python3.6-nginx.git](https://gitlab.com/imp555/python3.6-nginx.git)
![62.png](https://qiita-image-store.s3.amazonaws.com/0/324255/1ebe7b02-7e4d-efd0-8e10-45c36168cb66.png)

![63.png](https://qiita-image-store.s3.amazonaws.com/0/324255/d52b26de-d33a-9350-20ff-0914911a02fc.png)
Buildの様子を見に行きましょう。
Builds > Builds から Last Buildの#1をクリックし、Logsタブをクリックします。
![64.png](https://qiita-image-store.s3.amazonaws.com/0/324255/29f49e1a-5df0-afac-4951-d2e73a031f38.png)
しばらく待つとこのように`Pull Successful`と表示され完成です。
これでプロジェクト内のImage Streamに`python3.6-nginx:latest`が登録されました。
![65.png](https://qiita-image-store.s3.amazonaws.com/0/324255/7512e74d-423c-c396-5314-1a09a227afaf.png)
## いよいよDjango2の出番。`django2-nginx-ex:latest`をビルドする
もう一度Overviewに戻り、今度はDjango2ソースがあるGitから`openshift/templates/django2-nginx.json`をRAWでCopyしインポートします。
Git: [https://gitlab.com/imp555/django2-nginx.git](https://gitlab.com/imp555/django2-nginx.git)
![66.png](https://qiita-image-store.s3.amazonaws.com/0/324255/8e7c436d-31a4-9750-84bb-4ad5cd32b7d5.png)
![67.png](https://qiita-image-store.s3.amazonaws.com/0/324255/8efd3631-485d-6d90-070f-84d7e4f738c3.png)
ここでテンプレートを登録するか聞かれますがOffのままにしておきます。
![68.png](https://qiita-image-store.s3.amazonaws.com/0/324255/b831186e-5144-c58d-0038-0785bafd6164.png)
ほぼ標準の値でいけるのですが、一か所だけ修正します。
それは`Namespace`です。
なぜならば`python3.6-nginx:latest`を登録したのは自分のProject内だからです。
![69.png](https://qiita-image-store.s3.amazonaws.com/0/324255/11623658-1495-16ae-a26a-ba794b9ab37c.png)
![70.png](https://qiita-image-store.s3.amazonaws.com/0/324255/87026ce8-ca8b-18c9-f44e-97ce2635fb58.png)
あとは先ほどと同様Buildの様子をうかがいに行きます。
![71.png](https://qiita-image-store.s3.amazonaws.com/0/324255/31db6b1c-79b6-c3e1-16eb-8ae63974e6b3.png)
ここでまたもやminishift Web-Consoleがだんまり決め込みました。NotePCではリソース不足かな...(笑)
minishift stop && minishift start
![72.png](https://qiita-image-store.s3.amazonaws.com/0/324255/ee9809be-1ea5-8c74-1873-1f1928b7237d.png)
`Generic build failed`となっているのでNameをクリックしStart Buildをクリックします。
![73.png](https://qiita-image-store.s3.amazonaws.com/0/324255/4995d0a6-04e5-b76f-02fa-88f4a4291d70.png)
minishiftのログをターミナルで確認すると`connection refused`多発(涙)
何度やっても駄目なら仮想マシン再起動→PC再起動ですね(笑)
全てのレイヤーでPushが成功したらいよいよPodが作られます。 
![74.png](https://qiita-image-store.s3.amazonaws.com/0/324255/828598ee-172c-5d24-4d70-07625b540846.png)
## Podの動作確認
Applications > PodsからRunnningのPodをクリックし、Logsをクリックします。
![75.png](https://qiita-image-store.s3.amazonaws.com/0/324255/1dc72e63-768f-438e-0f15-6d2a2cf2ea49.png)
![76.png](https://qiita-image-store.s3.amazonaws.com/0/324255/43a7e725-3435-e2ad-ac7a-0e8f4d60fdae.png)

成功しているとこんな感じでrunスクリプトが呼ばれた後、readinessProbe、livenessProbeのアクセスログが出力されます。

```
﻿===> starting run script ...
---> Migrating database ...
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying sessions.0001_initial... OK
---> Running nginx front-end server ...
---> Serving application with gunicorn (wsgi) ...
[2019-01-03 03:10:47 +0900] [29] [INFO] Starting gunicorn 19.9.0
[2019-01-03 03:10:47 +0900] [29] [INFO] Listening at: http://0.0.0.0:8080  (29)
[2019-01-03 03:10:47 +0900] [29] [INFO] Using worker: sync
[2019-01-03 03:10:47 +0900] [34] [INFO] Booting worker with pid: 34
[2019-01-03 03:10:48 +0900] [36] [INFO] Booting worker with pid: 36
[2019-01-03 03:10:48 +0900] [38] [INFO] Booting worker with pid: 38
[2019-01-03 03:10:48 +0900] [39] [INFO] Booting worker with pid: 39
127.0.0.1 - - [03/Jan/2019:03:10:54 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 409759 -
127.0.0.1 - - [03/Jan/2019:03:11:04 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 29514 -
```

Overviewの画面からURLをクリックするとPodにアクセスできます。
![77.png](https://qiita-image-store.s3.amazonaws.com/0/324255/d8929d55-ef58-f9ce-13ac-d3edbd7a8343.png)
これはHealthの画面です。
![78.png](https://qiita-image-store.s3.amazonaws.com/0/324255/de123410-99ff-002c-3d80-d4c3da63c88c.png)
URLの後ろに`api/`を付けるとじゃじゃん。今の時刻がJSON形式で出力されます。
![79.png](https://qiita-image-store.s3.amazonaws.com/0/324255/e6c9b7bb-bf3b-a80f-d8dc-8e9caf605d0b.png)
HTTPヘッダーも確認します。バージョン番号が秘匿されていますね♪
![80.png](https://qiita-image-store.s3.amazonaws.com/0/324255/fe00db42-9945-bf3b-9fca-e34af1331012.png)
PodのLogsタブに戻るとアクセスログが出力されていますね。

```
127.0.0.1 - - [03/Jan/2019:03:19:04 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 8187 -
127.0.0.1 - - [03/Jan/2019:03:19:14 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1070 -
127.0.0.1 - - [03/Jan/2019:03:19:18 +0900] django2-nginx-ex-django2-nginx.192.168.42.246.nip.io "GET / HTTP/1.0" 200 0 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" 192.168.42.1, 172.17.0.1 - 0 1124 -
127.0.0.1 - - [03/Jan/2019:03:19:24 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1438 -
127.0.0.1 - - [03/Jan/2019:03:19:34 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1584 -
127.0.0.1 - - [03/Jan/2019:03:19:44 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1064 -
127.0.0.1 - - [03/Jan/2019:03:19:54 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 18799 -
127.0.0.1 - - [03/Jan/2019:03:20:04 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 61732 -
127.0.0.1 - - [03/Jan/2019:03:20:14 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 2310 -
127.0.0.1 - - [03/Jan/2019:03:20:24 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1083 -
127.0.0.1 - - [03/Jan/2019:03:20:34 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 5952 -
127.0.0.1 - - [03/Jan/2019:03:20:44 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1038 -
127.0.0.1 - - [03/Jan/2019:03:20:54 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1008 -
127.0.0.1 - - [03/Jan/2019:03:21:02 +0900] django2-nginx-ex-django2-nginx.192.168.42.246.nip.io "GET /api/ HTTP/1.0" 200 37 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" 192.168.42.1, 172.17.0.1 - 37 7424 -
127.0.0.1 - - [03/Jan/2019:03:21:04 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 155066 -
127.0.0.1 - - [03/Jan/2019:03:21:14 +0900] 172.17.0.12:8000 "GET / HTTP/1.0" 200 0 "-" "kube-probe/1.11+" 172.17.0.1 - 0 1347 -
```

これで一応目標達成です♪

# gunicornのチューニング
gunicornのチューニングはGitの`conf/gunicorn.py`で行います。

```text:gitlab.com/imp555/django2-nginx.git
├─conf
│  └─gunicorn.py
```

```python:conf/gunicorn.py
"""
You can set APP_CONFIG to point to this file to enable automatic reloading of
modules.
"""

# ===============================================================================
# gunicorn settings
# http://docs.gunicorn.org/en/stable/settings.html
# ===============================================================================

# ------------------------------------------------------------------------------
# Debugging
# ------------------------------------------------------------------------------

reload = True

# ------------------------------------------------------------------------------
# Logging
# ------------------------------------------------------------------------------

# gunicorn original access log format
# access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

access_log_format = '%(h)s %(l)s %(u)s %(t)s %({Host}i)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s" %({X-Forwarded-For}i)s %(l)s %(b)s %(D)s %(l)s'

# gunicorn error log to stderr
errorlog = '-'

# ------------------------------------------------------------------------------
# Worker Processes
# ------------------------------------------------------------------------------
timeout = 30
keepalive = 3

```


# nginxのチューニング
nginxのチューニングはGitの`etc/nginx/nginx.conf`および`etc/nginx/conf.d/default.conf`で行います。

## nginx.confのチューニング

```text:gitlab.com/imp555/django2-nginx.git
├─etc
│   └─nginx
│       ├─nginx.conf 
│       └─conf.d
│           └─default.conf 
```

```text:etc/nginx/nginx.conf

#user  nginx;
worker_processes  2;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    server_tokens off;
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    # access_log  /var/log/nginx/access.log  main;
    access_log off;

    sendfile        on;
    tcp_nopush      on;

    keepalive_timeout 65;

    gzip  on;
    gzip_types image/png image/gif image/jpeg text/javascript text/css;
    gzip_min_length 1000;
    gzip_proxied any;
    gzip_comp_level 9;

    upstream app_server {
        server 127.0.0.1:8080 fail_timeout=0;
    }

    include /etc/nginx/conf.d/*.conf;
}
```
## default.confのチューニング

```text:etc/nginx/conf.d/default.conf
server {
    listen  8000;
    server_name localhost;

    location /static {
        alias /opt/app-root/src/static;
    }

    location /favicon {
      empty_gif;
      access_log    off;
      log_not_found off;
    }

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

# SampleのDjango2 Projectを変更するには
SampleでGitに上がっているDjango2のProjectは簡単な現在の時刻を返すAPIのみ
このSampleのProjectを入れ替えるには以下の手順で行います。

1. Git([https://gitlab.com/imp555/django2-nginx.git](https://gitlab.com/imp555/django2-nginx.git))をCloneする
2. Sample Projectである次のファイルを削除する。(`sample/`, `restapi/`, `static/*`, `manage.py`, `wsgi.py`)
3. 入れたいDjango2のProjectをCopyしてくる
4. `wsgi.py`はProjectのルートフォルダからGitのルートへコピーしてくる
5. requirements.txtに必要なPythonモジュールを列記する。(gunicornはそのまま)
6. gunicorn,nginxの設定修正がある場合はそれぞれ行う
7. 別のGitへPush
8. OverviewからTemplateをImportし、聞かれる`Git Repository URL`を7.のGitに向ける。(ブランチがある場合は、`Git Reference`も併せて修正)　※`openshift/templates/django2-nginx.json`のGit URLを変更するのも可
![90.png](https://qiita-image-store.s3.amazonaws.com/0/324255/36000c1d-9819-3bc6-5c16-f2e5bed30b4f.png)
9. Build

## 注意点
- 制的コンテンツは`static`フォルダ内に登録すること

# 参考にしたもの
https://github.com/sclorg/s2i-python-container.git
https://github.com/openshift/django-ex.git

# 今回のGit
一応、何かあれば対応するつもりではいますが、そのスピード感は？です(笑)
- `python3.6-nginx:latest` … [https://gitlab.com/imp555/python3.6-nginx.git](https://gitlab.com/imp555/python3.6-nginx.git)
- `django2-nginx-ex:latest` … [https://gitlab.com/imp555/django2-nginx.git](https://gitlab.com/imp555/django2-nginx.git)