Project: Athenz (Provisioning / Key Management)
File: deploy-using-helm.md

<!-- TOC depthFrom:2 updateOnSave:true --> - - - - - - - <!-- /TOC --> <a id="markdown-note" name="note"></a> This procedure is for DEV env. ONLY. Please review every steps with your security policies before deploying to the production environment. This procedure works on `default` namespace ONLY. <a id="markdown-prerequisites" name="prerequisites"></a> `helm` `kubectl` <a id="markdown-steps" name="steps"></a> <a id="markdown-0-set-up-env-for-the-following-steps" name="0-set-up-env-for-the-following-steps"></a> Extra prerequisites: `bash`, `python`, `openssl`, `curl` ```bash export BASE_DIR="$(git rev-parse --show-toplevel)" . "${BASE_DIR}/docker/env.sh" . "${DOCKER_DIR}/sample/env.dev.sh" export WORKSPACE="${BASE_DIR}/kubernetes/docs/sample" export ZMSHELMFILE="${BASE_DIR}/kubernetes/docs/sample/zms-files" export ZTSHELMFILE="${BASE_DIR}/kubernetes/docs/sample/zts-files" relpath() { python -c "import os.path,sys;print(os.path.relpath(sys.argv[1],sys.argv[2]))" "$1" "$2" } createrellink() { ln -sf "$(relpath "$1" "$2")" "$2" } alias admincurl="curl --silent --fail --show-error --cacert ${DEVATHENZCAPATH} --key ${DEVDOMAINADMINCERTKEYPATH} --cert ${DEVDOMAINADMINCERT_PATH}" ``` ```bash export DEVDOMAINADMIN='user.github-1234567' export HOSTEXTERNALIP='127.0.0.1' export ZMSRELEASENAME='dev-zms' export ZTSRELEASENAME='dev-zts' export ZMSURL="https://${HOSTEXTERNAL_IP}:8000" export ZTSURL="https://${HOSTEXTERNAL_IP}:8001" export ZMSHOST="${ZMSRELEASE_NAME}-athenz-zms.default.svc.cluster.local" export ZTSHOST="${ZTSRELEASE_NAME}-athenz-zts.default.svc.cluster.local" export ZMSDBADMIN_PASS=<your-password> export ZMSRODBADMIN_PASS=<your-password> export ZTSDBADMIN_PASS=<your-password> ``` <a id="markdown-1-prepare-the-docker-images" name="1-prepare-the-docker-images"></a> <a id="markdown-11-build-the-athenz-docker-images" name="11-build-the-athenz-docker-images"></a> To build the Athenz docker image, plx refer to . <a id="markdown-12-push-the-athenz-docker-images-to-your-own-repo" name="12-push-the-athenz-docker-images-to-your-own-repo"></a> ```bash ATHENZ_TAG=1.9.27 DOCKERREPONAME=athenz docker tag athenz-zms-server "${DOCKERREPONAME}/athenz-zms-server:${ATHENZ_TAG}" docker tag athenz-zts-server "${DOCKERREPONAME}/athenz-zts-server:${ATHENZ_TAG}" docker tag athenz-setup-env "${DOCKERREPONAME}/athenz-setup-env:${ATHENZ_TAG}" docker push "${DOCKERREPONAME}/athenz-zms-server:${ATHENZ_TAG}" docker push "${DOCKERREPONAME}/athenz-zts-server:${ATHENZ_TAG}" docker push "${DOCKERREPONAME}/athenz-setup-env:${ATHENZ_TAG}" ``` <a id="markdown-2-define-trust-of-your-deployment" name="2-define-trust-of-your-deployment"></a> For details, please refer to . ```bash sh "${DEVCADIR}/create-self-signed-ca.sh" find "${DEVCADIR}" -name '*ca.pem' | xargs -I openssl x509 -text -in _ | grep 'Issuer: ' sh "${DEVDOMAINADMIN_DIR}/create-self-signed-user-cert.sh" openssl x509 -text -noout -in "${DEVDOMAINADMINCERTPATH}" | grep -E 'Issuer:|Subject:|DNS:' ``` <a id="markdown-3-prepare-zms-credentials" name="3-prepare-zms-credentials"></a> For details, please refer to . ```bash sh "${DEVZMSDIR}/create-self-signed-certs.sh" openssl x509 -text -noout -in "${DEVZMSCERT_PATH}" | grep -E 'Issuer:|Subject:|DNS:' openssl ecparam -noout -genkey -name prime256v1 -out "${ZMSPRIVATEKEY_PATH}" openssl ec -pubout -in "${ZMSPRIVATEKEYPATH}" -out "${ZMSPUBLICKEYPATH}" mkdir -p "${ZMSHELMFILE}/secrets/tls/CAs" createrellink "${DEVCADIR}/athenzca.pem" "${ZMSHELM_FILE}/secrets/tls/CAs" createrellink "${DEVCADIR}/serviceca.pem" "${ZMSHELM_FILE}/secrets/tls/CAs" createrellink "${DEVCADIR}/userca.pem" "${ZMSHELM_FILE}/secrets/tls/CAs" createrellink "${DEVZMSCERTPATH}" "${ZMSHELM_FILE}/secrets/tls" createrellink "${DEVZMSCERTKEYPATH}" "${ZMSHELMFILE}/secrets/tls" createrellink "${ZMSPRIVATEKEYPATH}" "${ZMSHELM_FILE}/secrets" ``` <a id="markdown-4-prepare-zts-credentials" name="4-prepare-zts-credentials"></a> For details, please refer to . ```bash sh "${DEVZTSDIR}/create-self-signed-certs.sh" openssl x509 -text -noout -in "${DEVZTSCERT_PATH}" | grep -E 'Issuer:|Subject:|DNS:' openssl x509 -text -noout -in "${DEVZTSSIGNERCERTPATH}" | grep -E 'Issuer:|Subject:|DNS:' openssl x509 -text -noout -in "${DEVZMSCLIENTCERTPATH}" | grep -E 'Issuer:|Subject:|DNS:' openssl ecparam -noout -genkey -name prime256v1 -out "${ZTSPRIVATEKEY_PATH}" openssl ec -pubout -in "${ZTSPRIVATEKEYPATH}" -out "${ZTSPUBLICKEYPATH}" mkdir -p "${ZTSHELMFILE}/secrets/tls/CAs" mkdir -p "${ZTSHELMFILE}/secrets/signer" mkdir -p "${ZTSHELMFILE}/secrets/zms-client" createrellink "${DEVCADIR}/athenzca.pem" "${ZTSHELM_FILE}/secrets/tls/CAs" createrellink "${DEVCADIR}/serviceca.pem" "${ZTSHELM_FILE}/secrets/tls/CAs" createrellink "${DEVCADIR}/userca.pem" "${ZTSHELM_FILE}/secrets/tls/CAs" createrellink "${DEVZTSCERTPATH}" "${ZTSHELM_FILE}/secrets/tls" createrellink "${DEVZTSCERTKEYPATH}" "${ZTSHELMFILE}/secrets/tls" createrellink "${DEVZTSSIGNERCERTPATH}" "${ZTSHELMFILE}/secrets/signer" createrellink "${DEVZTSSIGNERCERTKEYPATH}" "${ZTSHELM_FILE}/secrets/signer" createrellink "${DEVZMSCLIENTCERTBUNDLEPATH}" "${ZTSHELM_FILE}/secrets/zms-client" createrellink "${DEVZMSCLIENTCERTKEYPATH}" "${ZTSHELM_FILE}/secrets/zms-client" createrellink "${ZTSPRIVATEKEYPATH}" "${ZTSHELM_FILE}/secrets" ``` <a id="markdown-5-setup-zms-db" name="5-setup-zms-db"></a> To setup a database, please refer to . Schema SQL file: `curl "https://raw.githubusercontent.com/AthenZ/athenz/v${ATHENZTAG}/servers/zms/schema/zmsserver.sql"` Database name: `zms_server` Database user: `zms_admin` ```bash mysql -u root mysql -u zms_admin ``` Update corresponding ZMS properties: update after `helm pull` (see below) <a id="markdown-6-deploy-zms" name="6-deploy-zms"></a> ```bash helm pull athenz-zms -d "${WORKSPACE}" --untar \ --repo "https://raw.githubusercontent.com/AthenZ/athenz/v${ATHENZ_TAG}/kubernetes/charts" vi "${WORKSPACE}/athenz-zms/files/conf/zms.properties" ln -sf "${ZMSHELMFILE}/secrets" "${WORKSPACE}/athenz-zms/files/" grep -B 1 -A 6 'registry: ' "${BASE_DIR}/kubernetes/docs/sample/dev-zms-values.yaml" helm upgrade --install "${ZMSRELEASENAME}" "${WORKSPACE}/athenz-zms" \ --set "password.jdbc=${ZMSDBADMIN_PASS}" \ --set

\ --set "password.keystore=${ZMSKEYSTOREPASS}" \ --set "password.truststore=${ZMSTRUSTSTOREPASS}" \ -f "${BASE_DIR}/kubernetes/docs/sample/dev-zms-values.yaml" ``` ```bash while true; do kubectl port-forward --address 0.0.0.0 "$(kubectl get pod -l app=athenz-zms -o jsonpath='{.items[0].metadata.name}')" 8000:4443; done ``` ```bash admincurl --request GET --url "${ZMSURL}/zms/v1/status" | jq admincurl --request GET --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zms" | jq admincurl --request GET --url "${ZMSURL}/zms/v1/principal" | jq '.service' ``` ```bash kubectl describe $(kubectl get pod -l "app=athenz-zms" -o name) kubectl logs --tail=50 --all-containers $(kubectl get pod -l "app=athenz-zms" -o name) --timestamps=true | sort kubectl get events --sort-by=.metadata.creationTimestamp kubectl exec -it $(kubectl get pod -l "app=athenz-zms" -o name) -- /bin/sh less /opt/athenz/zms/logs/zms_server/server.log ``` ```bash rm -rf "${WORKSPACE}/athenz-zms"* helm uninstall "${ZMSRELEASENAME}" ``` <a id="markdown-7-register-zts-services-key-to-zms" name="7-register-zts-services-key-to-zms"></a> ```bash ENCODEDZTSPUBLICKEY="$(cat "${ZTSPUBLICKEYPATH}" | base64 -b 0 | tr '\+\=\/' '\.\-\_')" DATA='{"name": "sys.auth.zts","publicKeys": [{"id": "0","key": "'"${ENCODEDZTSPUBLIC_KEY}"'"}]}' admincurl --request DELETE -D - --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zts" admincurl --request PUT -D - --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zts" \ --header 'content-type: application/json' \ --data "${DATA}" admincurl --request GET --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zts" | jq ``` ```bash ENCODEDZMSPUBLICKEY="$(cat "${ZMSPUBLICKEYPATH}" | base64 -b 0 | tr '\+\=\/' '\.\-\_')" DATA='{"name": "sys.auth.zms","publicKeys": [{"id": "0","key": "'"${ENCODEDZMSPUBLIC_KEY}"'"}]}' admincurl --request DELETE -D - --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zms" admincurl --request PUT -D - --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zms" \ --header 'content-type: application/json' \ --data "${DATA}" admincurl --request GET --url "${ZMSURL}/zms/v1/domain/sys.auth/service/zms" | jq ``` <a id="markdown-8-generate-athenzconfjson" name="8-generate-athenzconfjson"></a> Download athenz-utils-${ATHENZ_TAG}-bin.tar.gz from (click on the `Browse` button, choose the latest version directory). ```bash tar xvfz athenz-utils-${ATHENZ_TAG}-bin.tar.gz cp athenz-utils-${ATHENZ_TAG}/bin/<PLATFORM>/athenz-conf ./ mkdir -p "${ZTSHELMFILE}/conf" ./athenz-conf -c "${DEVATHENZCA_PATH}" \ -svc-key-file "${DEVDOMAINADMINCERTKEY_PATH}" \ -svc-cert-file "${DEVDOMAINADMINCERTPATH}" \ -o "${ZTSHELMFILE}/conf/athenz_conf.json" \ -t "https://${ZTS_HOST}:8443" \ -z "${ZMS_URL}" sed -i '' "s,${ZMSURL},https://${ZMSHOST}:4443," "${ZTSHELMFILE}/conf/athenz_conf.json" less "${ZTSHELMFILE}/conf/athenz_conf.json" ``` <a id="markdown-9-setup-zts-db" name="9-setup-zts-db"></a> To setup a database, please refer to . The differences between ZMS and ZTS are: Schema SQL file: `curl "https://raw.githubusercontent.com/AthenZ/athenz/v${ATHENZTAG}/servers/zts/schema/ztsserver.sql"` Database name: `zts_store` Database user: `zts_admin` ```bash mysql -u root mysql -u zts_admin ``` Update corresponding ZTS properties: update after `helm pull` (see below) <a id="markdown-10-deploy-zts" name="10-deploy-zts"></a> ```bash helm pull athenz-zts -d "${WORKSPACE}" --untar \ --repo "https://raw.githubusercontent.com/AthenZ/athenz/v${ATHENZ_TAG}/kubernetes/charts" vi "${WORKSPACE}/athenz-zts/files/conf/zts.properties" ln -sf "${ZTSHELMFILE}/conf/athenz_conf.json" "${WORKSPACE}/athenz-zts/files/conf/" ln -sf "${ZTSHELMFILE}/secrets" "${WORKSPACE}/athenz-zts/files/" grep -B 1 -A 6 'registry: ' "${BASE_DIR}/kubernetes/docs/sample/dev-zts-values.yaml" helm upgrade --install "${ZTSRELEASENAME}" "${WORKSPACE}/athenz-zts" \ --set "password.jdbc=${ZTSDBADMIN_PASS}" \ --set "password.keystore=${ZTSKEYSTOREPASS}" \ --set "password.truststore=${ZTSTRUSTSTOREPASS}" \ --set "password.signerKeystore=${ZTSSIGNERKEYSTORE_PASS}" \ --set "password.signerTruststore=${ZTSSIGNERTRUSTSTORE_PASS}" \ --set "password.zmsClientKeystore=${ZMSCLIENTKEYSTORE_PASS}" \ --set "password.zmsClientTruststore=${ZMSCLIENTTRUSTSTORE_PASS}" \ -f "${BASE_DIR}/kubernetes/docs/sample/dev-zts-values.yaml" ``` ```bash while true; do kubectl port-forward --address 0.0.0.0 "$(kubectl get pod -l app=athenz-zts -o jsonpath='{.items[0].metadata.name}')" 8001:8443; done ``` ```bash admincurl --request GET --url "${ZTSURL}/zts/v1/status" | jq admincurl --request GET --url "${ZTSURL}/zts/v1/domain/sys.auth/service/zts" | jq admincurl --request GET --url "${ZTSURL}/zts/v1/domain/sys.auth/token?role=admin" | jq '.token' ``` ```bash kubectl describe $(kubectl get pod -l "app=athenz-zts" -o name) kubectl logs --tail=100 --all-containers $(kubectl get pod -l "app=athenz-zts" -o name) --timestamps=true | sort kubectl get events --sort-by=.metadata.creationTimestamp kubectl exec -it $(kubectl get pod -l "app=athenz-zts" -o name) -- /bin/sh less /opt/athenz/zts/logs/zts_server/server.log ``` ```bash rm -rf "${WORKSPACE}/athenz-zts"* helm uninstall "${ZTSRELEASENAME}" ```