Repository: minio/minio (language: go)

Issue:
[helm-chart] Race condition when creating users/policies via chart
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->

When I install the fresh helm chart, a policy is created, then the user is assigned that policy.

## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->

I believe this race condition is most common when installing a fresh chart.

If the `minio-make-user` container finishes before `minio-make-policy` a user is created, but no policy is assigned:

```
kubectl logs -n minio minio-post-job-kpq6v -c minio-make-user
Connecting to MinIO server: http://minio:9000
mc: <ERROR> Unable to initialize new alias from the provided credentials. Get "http://minio:9000/probe-bucket-sign-vz1kivnhnzn4/?location=": dial tcp 10.120.143.162:9000: connect: connection refused.
"Failed attempts: 1"
mc: <ERROR> Unable to initialize new alias from the provided credentials. Get "http://minio:9000/probe-bucket-sign-ky499ni4djfc/?location=": dial tcp 10.120.143.162:9000: connect: connection refused.
"Failed attempts: 2"
Added `myminio` successfully.
Creating user 'loki'
Added user `loki` successfully.
Adding policy 'loki' for 'loki'
mc: <ERROR> Unable to make user/group policy association. The canned policy does not exist. (Specified canned policy does not exist).
```

```
kubectl logs -n minio minio-post-job-kpq6v -c minio-make-policy
Connecting to MinIO server: http://minio:9000
mc: <ERROR> Unable to initialize new alias from the provided credentials. Get "http://minio:9000/probe-bucket-sign-kgfkxlxjoz6l/?location=": dial tcp 10.120.143.162:9000: connect: connection refused.
"Failed attempts: 1"
Added `myminio` successfully.
Checking policy: foo:admins (in /config/policy_0.json)
Creating policy 'foo:admins'
Created policy `foo:admins` successfully.
Checking policy: foo/admins (in /config/policy_1.json)
Creating policy 'foo/admins'
Created policy `foo/admins` successfully.
Checking policy: loki (in /config/policy_2.json)
Creating policy 'loki'
Created policy `loki` successfully.
```

When I re-deploy the helm chart, the user's policy is correctly associated!

## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
<!--- or ideas how to implement the addition or change -->

Maybe the job could use `initContainers` to run these jobs serially, so that we always run `minio-make-policy` before `minio-make-user`?

I'm not sure if there is any association between the `minio-make-bucket` and the others.

If you'd like me to open a PR I'm willing to but would want some guidance on the desired behavior of `minio-make-bucket` 🚀 

## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->

1. Install the minio helm chart `5.0.11` with this values file:
   ```yaml
   rootUser: "admin"
   rootPassword: "admin123"

   fullnameOverride: minio

   # This is my single node instance, run in standalone mode. See foo/minio for the
   # full distributed setup.
   mode: standalone

   securityContext:
     # This is used to auto chown -R /chmod -R the "local" volume. It won't work
     # with "hostPath" types. Without this setting the chown/chmod will happen
     # *every* restart. With OnRootMismatch it will only run when a new volume is
     # setup, like when we delete volumes or move to a new node.
     fsGroupChangePolicy: OnRootMismatch

   persistence:
     enabled: true
     storageClass: "local-path"

   # This is the api ingress. This is where you'll point the "mc" client at.
   ingress:
     enabled: true
     path: /
     hosts:
       - minio.dev.example.com

   resources:
     requests:
       memory: 2Gi

   # This is the web UI. It's only used for humans to manage MinIO.
   consoleIngress:
     enabled: true
     path: /
     hosts:
       - minio-console.dev.example.com

   environment:
     # OIDC Settings
     MINIO_IDENTITY_OPENID_CONFIG_URL: https://dex.dev.example.com/.well-known/openid-configuration
     MINIO_IDENTITY_OPENID_CLIENT_ID: minio
     MINIO_IDENTITY_OPENID_CLAIM_NAME: groups
     # MINIO_IDENTITY_OPENID_CLAIM_PREFIX: ""
     MINIO_IDENTITY_OPENID_SCOPES: openid,profile,email,groups
     MINIO_IDENTITY_OPENID_REDIRECT_URI: https://minio-console.dev.example.com/oauth_callback

   policies:
     - name: foo:admins # GitHub OIDC login
       statements:
         - actions:
             - "admin:*"
         - actions:
             - "s3:*"
           resources:
             - 'arn:aws:s3:::*'
     - name: foo/admins # Gitlab OIDC login
       statements:
         - actions:
             - "admin:*"
         - actions:
             - "s3:*"
           resources:
             - 'arn:aws:s3:::*'
     - name: loki
       statements:
         - actions:
             - "s3:*"
           resources:
             - 'arn:aws:s3:::loki-chunks/*'
             - 'arn:aws:s3:::loki-ruler/*'
             - 'arn:aws:s3:::loki-admin/*'

   users:
     - accessKey: loki
       secretKey: lokiloki
       policy: loki

   buckets:
     # NOTE: These are single-node so I don't specify versioning at all.
     - name: thanos
       policy: none
       purge: false
     - name: loki-chunks
       policy: none
       purge: false
     - name: loki-ruler
       policy: none
       purge: false
     - name: loki-admin
       policy: none
       purge: false
     - name: testa
       policy: none
       purge: false
     - name: testb
       policy: none
       purge: false
     - name: testc
       policy: none
       purge: false
     - name: testd
       policy: none
       purge: false
   ```
2. Check the `loki` user, I used the UI but you could also use `mc` to see it's missing a policy. I believe this is a race condition based on how long containers take to pull, so it might not happen if the stars do or don't align correctly.

## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->

I'm re-installing minio (single node) in my lab and am starting totally from scratch. I noticed this on a fresh install.

Fixing this will help users who are installing MinIO in a lab/dev/preproduction environment re-create instances.

## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->

I don't think so, I haven't been using the helm chart based policies very long.

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio --version`):
* Server setup and configuration:
* Operating System and version (`uname -a`):

```
bash-4.4$ minio --version
minio version RELEASE.2023-06-19T19-52-50Z (commit-id=f9b8d1c6999e65ab31899cbbe0314f5a4e5257c3)
Runtime: go1.19.10 linux/amd64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-2023 MinIO, Inc.
```

configuration attached above (helm chart)

OS: Kubernetes (kubeadm provisioned) 1.26.6, Ubuntu, kernel:

```
uname -r
5.15.0-75-generic
```


Discussion / hints:
@fernferret feel free to send a PR to fix the issue. 

Fix patch:
diff --git a/helm/minio/templates/post-job.yaml b/helm/minio/templates/post-job.yaml
index b6a5e18dd040f..99b6997b22dca 100644
--- a/helm/minio/templates/post-job.yaml
+++ b/helm/minio/templates/post-job.yaml
@@ -85,6 +85,40 @@ spec:
       {{- if .Values.serviceAccount.create }}
       serviceAccountName: {{ .Values.serviceAccount.name }}
       {{- end }}
+      { { - if .Values.policies } }
+      initContainers:
+        - name: minio-make-policy
+          image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
+          { { - if .Values.makePolicyJob.securityContext.enabled } }
+          securityContext:
+            runAsUser: { { .Values.makePolicyJob.securityContext.runAsUser } }
+            runAsGroup: { { .Values.makePolicyJob.securityContext.runAsGroup } }
+          { { - end } }
+          imagePullPolicy: { { .Values.mcImage.pullPolicy } }
+          { { - if .Values.makePolicyJob.exitCommand } }
+          command: [ "/bin/sh", "-c" ]
+          args: [ "/bin/sh /config/add-policy; EV=$?; {{ .Values.makePolicyJob.exitCommand }} && exit $EV" ]
+          { { - else } }
+          command: [ "/bin/sh", "/config/add-policy" ]
+          { { - end } }
+          env:
+            - name: MINIO_ENDPOINT
+              value: { { template "minio.fullname" . } }
+            - name: MINIO_PORT
+              value: { { .Values.service.port | quote } }
+          volumeMounts:
+            - name: etc-path
+              mountPath: /etc/minio/mc
+            - name: tmp
+              mountPath: /tmp
+            - name: minio-configuration
+              mountPath: /config
+            { { - if .Values.tls.enabled } }
+            - name: cert-secret-volume-mc
+              mountPath: { { .Values.configPathmc } }certs
+            {{- end }}
+          resources: { { - toYaml .Values.makePolicyJob.resources | nindent 12 } }
+      { { - end } }
       containers:
         {{- if .Values.buckets }}
         - name: minio-make-bucket
@@ -152,39 +186,6 @@ spec:
             {{- end }}
           resources: {{- toYaml .Values.makeUserJob.resources | nindent 12 }}
         {{- end }}
-        {{- if .Values.policies }}
-        - name: minio-make-policy
-          image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
-          {{- if .Values.makePolicyJob.securityContext.enabled }}
-          securityContext:
-            runAsUser: {{ .Values.makePolicyJob.securityContext.runAsUser }}
-            runAsGroup: {{ .Values.makePolicyJob.securityContext.runAsGroup }}
-          {{- end }}
-          imagePullPolicy: {{ .Values.mcImage.pullPolicy }}
-          {{- if .Values.makePolicyJob.exitCommand }}
-          command: [ "/bin/sh", "-c" ]
-          args: [ "/bin/sh /config/add-policy; EV=$?; {{ .Values.makePolicyJob.exitCommand }} && exit $EV" ]
-          {{- else }}
-          command: [ "/bin/sh", "/config/add-policy" ]
-          {{- end }}
-          env:
-            - name: MINIO_ENDPOINT
-              value: {{ template "minio.fullname" . }}
-            - name: MINIO_PORT
-              value: {{ .Values.service.port | quote }}
-          volumeMounts:
-            - name: etc-path
-              mountPath: /etc/minio/mc
-            - name: tmp
-              mountPath: /tmp
-            - name: minio-configuration
-              mountPath: /config
-            {{- if .Values.tls.enabled }}
-            - name: cert-secret-volume-mc
-              mountPath: {{ .Values.configPathmc }}certs
-            {{- end }}
-          resources: {{- toYaml .Values.makePolicyJob.resources | nindent 12 }}
-        {{- end }}
         {{- if .Values.customCommands }}
         - name: minio-custom-command
           image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"


Test patch: