Actions Runner Controllerを動かそうとして結局dindになった話

[Actions Runner Controller](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller)を諸事情から導入し、諸事情から最近のARCが使えずレガシー版のARCを使うことになった。
その備忘録

## まずはdindで試す

なにはともあれEKSで立てなくてはならない。ので
```bash
eksctl create cluster -f cluster.yaml
aws eks update-kubeconfig --name arc-cluster --region ap-northeast-1
```

```yaml:cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: arc-cluster
  region: ap-northeast-1


managedNodeGroups:
- name: arc-cluster-node
  instanceTypes:
    - t3.large
    - t3a.large
  spot: true
iam:
  withOIDC: true
  serviceAccounts: 
  - metadata:
      name: runner-service-account
      namespace: default
    attachPolicyARNs:
    - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser
  - metadata:
      name: ebs-csi-controller-sa
      namespace: default
    attachPolicyARNs:
    - arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy
    - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser
```

ebs-csi-controller-saはこの後使う。

## 手順通りにhelmでARCをデプロイ

```bash
helm repo add jetstack https://charts.jetstack.io
helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.13.3 \
  --set installCRDs=true

kubectl create namespace actions-runner-system
kubectl create secret generic controller-manager \
  --namespace actions-runner-system \
  --from-literal=github_app_id=10 \
  --from-literal=github_app_installation_id=10 \
  --from-file=github_app_private_key=./key.pem


helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
helm upgrade -f values.yaml --install --namespace actions-runner-system \
             --wait actions-runner-controller actions-runner-controller/actions-runner-controller \
             --set=authSecret.enabled=true

kubectl apply -f role.yaml
kubectl apply -f runner.yaml
```

```yaml:values.yaml
replicaCount: 1
authSecret:
  enabled: true
  create: false
  name: "controller-manager"
```

```yaml:role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: runner-role
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "create", "delete"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["get", "create"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get", "list", "watch",]
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "create", "delete"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: runner-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: runner-role
subjects:
- kind: ServiceAccount
  name: runner-service-account
  namespace: default
```

```yaml:runner.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: runner
spec:
  replicas: 1
  template:
      image: summerwind/actions-runner-dind
      dockerdWithinRunnerContainer: true
      serviceAccountName: runner-service-account
      repository: owner/repo

```

これで普通に使えるようになった。

## それではkubernetesモードにしてみる

[公式の手順に従って](https://github.com/actions/actions-runner-controller/blob/master/docs/deploying-alternative-runners.md)runner.yamlを修正。

```bash
kubectl apply -f runner.yaml
```

```yaml:runner.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: runner
spec:
  replicas: 1
  template:
      containerMode: kubernetes
      serviceAccountName: runner-service-account
      workVolumeClaimTemplate:
        storageClassName: "my-dynamic-storage-class"
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
      repository: owner/repo
```

失敗。storage classを作ってくれるかと思ったら作ってくれなかったので手動で作る。

### せっかくなのでebsでstorage classを作る

先ほど使わなかったロールを使ってアドオンを入れる。
この時ebs-csi-controller-saのロールのConditionを消すなりしてロールを読めるようにする。

```bash
aws eks create-addon \
 --region ap-northeast-1 \
 --cluster-name arc-cluster \
 --addon-name aws-ebs-csi-driver \
 --service-account-role-arn {先ほどのebs-csi-controller-saのARN}
```

storage classの作成

```console
kubectl apply -f storage.yaml
```

```yaml:storage.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-sc
  resources:
    requests:
      storage: 4Gi
```

ロールも紐づけておく

```bash
kubectl apply -f role.yaml
```

```yaml:role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: runner-role
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "create", "delete"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["get", "create"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get", "list", "watch",]
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "create", "delete"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: runner-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: runner-role
subjects:
- kind: ServiceAccount
  name: runner-service-account
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ebs-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: runner-role
subjects:
- kind: ServiceAccount
  name: ebs-csi-controller-sa
  namespace: default
```

作成したstrage classにしてservice accountを変えてもう一度

```bash
kubectl apply -f runner.yaml
```

```yaml:runner.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: runner
spec:
  replicas: 1
  template:
      containerMode: kubernetes
      serviceAccountName: ebs-csi-controller-sa
      workVolumeClaimTemplate:
        storageClassName: ebs-sc
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
      repository: owner/repo
```


デプロイできた。が、Dockerを使うアクションで以下のエラーが発生。

> Error: Jobs without a job container are forbidden on this runner, please add a 'container:' to your job or contact your self-hosted runner administrator.

これのことなのか
https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#configuring-kubernetes-mode

しかしこれは最近のやつでレガシーモードのドキュメントではない

でも一応やってみる。

```bash
kubectl apply -f runner.yaml
```

```yaml:runner.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: runner
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: runner
          image: ghcr.io/actions/actions-runner:latest
          command: ["/home/runner/run.sh"]
          env:
            - name: ACTIONS_RUNNER_REQUIRE_JOB_CONTAINER
              value: "false"
      containerMode: kubernetes
      serviceAccountName: ebs-csi-controller-sa
      workVolumeClaimTemplate:
        storageClassName: ebs-sc
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
      repository: owner/repo
```

今度はpodが起動しない！？


> Events:
> Type     Reason                  Age   From                     Message
>  ----     ------                  ----  ----                     -------
>   Warning  FailedScheduling        13s   default-scheduler        0/1 nodes are available: waiting for ephemeral volume controller to create the persistentvolumeclaim "runner-9ks9p-mxpdl-work". preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
>   Normal   Scheduled               6s    default-scheduler        Successfully assigned default/runner-9ks9p-mxpdl to ip-192-168-92-72.us-west-2.compute.internal
>   Normal   SuccessfulAttachVolume  3s    attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-9ff95439-82d2-46be-bb84-c3dfc0f4bee0"

イベントから見てもpodが起動しない理由はわからず…

どうしてgithub enterprise3.7なんか使ってるんだ…


## 結論
諦めてdind構成でやりましたとさ
そもそもレガシー環境やらされるのもなあ
サポート切れてるし