ArgoCDのYamlをうまく管理する方法

# 課題

- ダウンロードしてきたYamlを変更する必要がある
    - Deploymentの設定
    - Endpoint
    - Auth
    - RBAC
- 変更するとVersion Upgrade時に同じ変更をまた手動でやるのはキツイ

# 基本方針: Kustomizeで管理

- BaseでVersion指定
- overlaysで必要に応じていじる
    - Deploymentをいじるけい
    - ELBにする
    - Ingressにする
    - Google OAuthを使う
    - RBACの設定
    - など

参照: https://argoproj.github.io/argo-cd/operator-manual/declarative-setup/#manage-argo-cd-using-argo-cd 

例:

`kustomization.yaml` では `resources`と`pathcesStrategicMerge`を使い分ける (`Secret`をRepoにいれたくなかったので今回は、[SealedSecret](https://github.com/bitnami-labs/sealed-secrets)を利用)

```yaml:kustomization.yaml
namespace: argocd
bases:
  - github.com/argoproj/argo-cd//manifests/cluster-install?ref=v1.6.0
patchesStrategicMerge:
  - argocd-notifications-cm.yaml
  - argocd-rbac-cm.yaml
  - argocd-cm.yaml
  - argocd-repo-server-deployment.yaml
  - argocd-dex-server-deployment.yaml
resources:
  - argocd-notifications-sealedsecret.yaml
  - argocd-google-oauth-sealedsecret.yaml
  - ingress.yaml
```

こうしておけば、Applyは以下のコマンドでいい

```
kubectl apply -k .
```

# カスタム設定

## 1. nodeSelector, priorityClassNameなど自分たちのKubernetesクラスタ上での管理設定をDeploymentに追加

ArgoCDに限らず普通のKustomizeの設定なので、Deploymentに対してOverlaysを準備してあげる

例: `nodeSelector`を追加して挙げる例

```yaml:argocd-server-deploymemt.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-server
spec:
  template:
    spec:
      containers:
        - name: argocd-server
          nodeSelector:
            disktype: ssd
```

```diff:kustomization.yaml
 namespace: argocd
 bases:
   - github.com/argoproj/argo-cd//manifests/cluster-install?ref=v1.6.0
 patchesStrategicMerge:
   - ...
+  - argocd-server-deploymemt.yaml
```
## 2. ELBを使う (AWSの話)

ELBかIngressどっちかでいい


serviceを追加して、`kustomization.yaml`に入れるだけでいい

```yaml:argocd-server-svc.yaml
apiVersion: v1
kind: Service
metadata:
    labels:
      app.kubernetes.io/component: server
      app.kubernetes.io/name: argocd-server
      app.kubernetes.io/part-of: argocd
    name: argocd-server
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:ap-northeast-1:xxxxxxxx:certificate/xxxxxxxxxxxxxxxxxxxxxxxxxx # httpsにする場合
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
      service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "2"
      service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "5"
      service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "2"
spec:
    type: LoadBalancer
    ports:
    - name: http
        port: 80
        protocol: TCP
        targetPort: 8080
    - name: https
        port: 443
        protocol: TCP
        targetPort: 8080
    selector:
      app.kubernetes.io/name: argocd-server
    loadBalancerSourceRanges:
      - xxx.xxx.xxx.xxx/32 # IP制限したい場合
```

```diff:kustomization.yaml
 namespace: argocd
 bases:
   - github.com/argoproj/argo-cd//manifests/cluster-install?ref=v1.6.0
 patchesStrategicMerge:
   - ...
+  - argocd-server-svc.yaml
```

## 3. Ingressにする 

参照: https://argoproj.github.io/argo-cd/operator-manual/ingress/
ELBかIngressどっちかでいい

- prerequisite: すでにingress controllerとexternal-dnsが存在
- 以下の2つのファイルをkustomization.yamlに追加する
- ハマリポイント: SSL終端をELBでやると [TOO_MANY_REDIRECTS](https://www.google.com/search?q=argocd+ingress+too+many+redirects&rlz=1C5CHFA_enJP915JP915&oq=argocd+too+many+redirects&aqs=chrome.1.69i57j0i22i30.4396j0j7&sourceid=chrome&ie=UTF-8) 問題に落ちいいるので、 `insecure` と `force-ssl-redirect` を `false` にした

```yaml:ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
  name: argocd
spec:
  rules:
  - host: argocd.example.com
    http:
      paths:
      - backend:
          serviceName: argocd-server
          servicePort: http
```

```yaml:argocd-server-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-server
spec:
  template:
    spec:
      containers:
        - name: argocd-server
          command:
            - argocd-server
            - --staticassets
            - /shared/app
            - --insecure # これが必要になる
```
## 4. Google Oauthを使う

参照: https://argoproj.github.io/argo-cd/operator-manual/user-management/google/

`GOOGLE_CLIENT_ID` と `GOOGLE_CLIENT_SECRET` は取得しておく

```yaml:argocd-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-cm
    app.kubernetes.io/part-of: argocd
  name: argocd-cm
  namespace: argocd
data:
  url: https://argocd.example.com

  dex.config: |
    connectors:
    - type: oidc
      id: google
      name: Google
      config:
        # Canonical URL of the provider, also used for configuration discovery.
        # This value MUST match the value returned in the provider config discovery.
        # See: https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfig
        issuer: https://accounts.google.com

        # Connector config values starting with a "$" will read from the environment.
        clientID: $GOOGLE_CLIENT_ID
        clientSecret: $GOOGLE_CLIENT_SECRET

        # Dex's issuer URL + "/callback"
        redirectURI: https://argocd.example.com/api/dex/callback

        # Google supports whitelisting allowed domains when using G Suite
        # (Google Apps). The following field can be set to a list of domains
        # that can log in:
        hostedDomains:
          - example.com
```

以下のsealedsecret.yamlはargocd-google-oauth-secret.yaml作成後に以下のコマンドで作成

```
kubeseal -o yaml < argocd-google-oauth-secret.yaml > argocd-google-oauth-sealedsecret.yaml
```

```yaml:argocd-google-oauth-sealedsecret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: argocd-google-oauth-secret
  namespace: argocd
spec:
  encryptedData:
    GOOGLE_CLIENT_ID: xxxxxxxxxxxxxxx
    GOOGLE_CLIENT_SECRET: xxxxxxxxxxxxxx
  template:
    metadata:
      creationTimestamp: null
      name: argocd-google-oauth-secret
      namespace: argocd
    type: Opaque
```

```yaml:argocd-dex-server-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-dex-server
spec:
  template:
    spec:
      containers:
      - name: dex
        envFrom:
          - secretRef:
              name: argocd-google-oauth-secret
```

```diff:kustomization.yaml
 namespace: argocd
 bases:
   - github.com/argoproj/argo-cd//manifests/cluster-install?ref=v1.6.0
 patchesStrategicMerge:
+  - argocd-cm.yaml
+  - argocd-dex-server-deployment.yaml
 resources:
+  - argocd-google-oauth-sealedsecret.yaml
```

## 5. RBACを使う

参照: https://argoproj.github.io/argo-cd/operator-manual/rbac/

```yaml:argocd-rbac-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-rbac-cm
    app.kubernetes.io/part-of: argocd
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.default: role:readonly
  scopes: '[email]' # google oauth使ってる場合
  policy.csv: |
    p, role:org-admin, applications, *, */*, allow
    p, role:org-admin, clusters, get, *, allow
    p, role:org-admin, repositories, get, *, allow
    p, role:org-admin, repositories, create, *, allow
    p, role:org-admin, repositories, update, *, allow
    p, role:org-admin, repositories, delete, *, allow

    g, yourname@test.com, role:org-admin
```

```diff:kustomization.yaml
 namespace: argocd
 bases:
   - github.com/argoproj/argo-cd//manifests/cluster-install?ref=v1.6.0
 patchesStrategicMerge:
+  - argocd-rbac-cm.yaml
```

# Upgrade version

1. `kustomization.yaml`で参照してるversionを変更
1. 必要に応じてdiffチェック
    ```
    kubectl diff -k .
    ```
1. apply
    ```
    kubectl apply -k .
    ```