Repository: fission/fission (language: go)

Issue:
v1.18.0, custom namespace, Waiting for CRDs to be installed
<!-- Please answer these questions before submitting your issue. Thanks! -->

<!-- Documentation URL: https://fission.io/docs -->
<!-- Troubleshooting guide: https://fission.io/docs/trouble-shooting/ -->

**Fission/Kubernetes version**

<!-- If you tested with other services, for example Istio, please also provide the version of service as well. -->

<pre>
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:29:58Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"linux/arm64"}
</pre>

**Kubernetes platform (e.g. Google Kubernetes Engine)**
docker-desktop

**Describe the bug**
<!--A clear and concise description of what the bug is.-->
Restarting pods when installed to custom namespace
<pre>
{"level":"error","ts":"2023-02-28T11:15:47.217Z","caller":"fission-bundle/main.go:304","msg":"mqt scaler manager exited","error":"error waiting for CRDs: timeout waiting for CRDs","errorVerbose":"timeout waiting for CRDs\nerror waiting for CRDs\ngithub.com/fission/fission/pkg/mqtrigger.StartScalerManager\n\tpkg/mqtrigger/scalermanager.go:166\nmain.runMQManager\n\tcmd/fission-bundle/main.go:83\nmain.main\n\tcmd/fission-bundle/main.go:302\nruntime.main\n\t/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/proc.go:250\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/asm_arm64.s:1172","stacktrace":"main.main\n\tcmd/fission-bundle/main.go:304\nruntime.main\n\t/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/proc.go:250"}
</pre>
**To Reproduce**

<!-- Please provide steps for reproducing the error. -->
<pre>
% export FISSION_NAMESPACE="first"

% kubectl create namespace $FISSION_NAMESPACE
namespace/first created

% kubectl create -k "github.com/fission/fission/crds/v1?ref=v1.18.0"
customresourcedefinition.apiextensions.k8s.io/canaryconfigs.fission.io created
customresourcedefinition.apiextensions.k8s.io/environments.fission.io created
customresourcedefinition.apiextensions.k8s.io/functions.fission.io created
customresourcedefinition.apiextensions.k8s.io/httptriggers.fission.io created
customresourcedefinition.apiextensions.k8s.io/kuberneteswatchtriggers.fission.io created
customresourcedefinition.apiextensions.k8s.io/messagequeuetriggers.fission.io created
customresourcedefinition.apiextensions.k8s.io/packages.fission.io created
customresourcedefinition.apiextensions.k8s.io/timetriggers.fission.io created

% helm upgrade --install --version v1.18.0 --namespace $FISSION_NAMESPACE fission fission-charts/fission-all --set defaultNamespace=$FISSION_NAMESPACE
Release "fission" does not exist. Installing it now.
NAME: fission
LAST DEPLOYED: Tue Feb 28 14:13:02 2023
NAMESPACE: first
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES: ....

</pre>
**Expected result**
<!--A clear and concise description of what you expected to happen.-->
All pods are up.

**Actual result**
Pods `kubewatcher`, `mqtrigger-keda`, `router`, `timer` restarts with same error.
Logs:
```
{"level":"info","ts":"2023-02-28T11:18:33.129Z","caller":"otel/provider.go:50","msg":"OTEL_EXPORTER_OTLP_ENDPOINT not set, skipping Opentelemtry tracing"}
{"level":"info","ts":"2023-02-28T11:18:33.130Z","caller":"crd/client.go:105","msg":"Waiting for CRDs to be installed"}
{"level":"error","ts":"2023-02-28T11:19:03.165Z","caller":"fission-bundle/main.go:304","msg":"mqt scaler manager exited","error":"error waiting for CRDs: timeout waiting for CRDs","errorVerbose":"timeout waiting for CRDs\nerror waiting for CRDs\ngithub.com/fission/fission/pkg/mqtrigger.StartScalerManager\n\tpkg/mqtrigger/scalermanager.go:166\nmain.runMQManager\n\tcmd/fission-bundle/main.go:83\nmain.main\n\tcmd/fission-bundle/main.go:302\nruntime.main\n\t/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/proc.go:250\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/asm_arm64.s:1172","stacktrace":"main.main\n\tcmd/fission-bundle/main.go:304\nruntime.main\n\t/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/proc.go:250"}
```




Discussion / hints:
@Tri0L I had a very similar issue, but with 1.17.0 and saw in two fission deployments. I suspect the namespace `fission` is still hardcoded somewhere in the yaml in 1.18.0. 
I will try 1.18 to see if I can see similar behaviour. 
This also happens when setting `FISSION_BUILDER_NAMESPACE` or `FISSION_FUNCTION_NAMESPACE` to any value.
I've struggling with this as well, even installing into "fission" namespace.

What I am finding is that unless I set one either the default namespace or one of the additional namespaces to "default", this fails.

No "default" namespace

```
2023-04-18T22:48:30.420Z    DEBUG    utils/namespace.go:49    namespaces    {"function_namespace": "", "builder_namespace": "", "default_namespace": "", "fission_resource_namespace": ["fission-ztps", "fission-default", "fission-datalake"]}
2023-04-18T22:48:30.431Z    INFO    otel/provider.go:50    OTEL_EXPORTER_OTLP_ENDPOINT not set, skipping Opentelemtry tracing
2023-04-18T22:48:30.431Z    INFO    crd/client.go:105    Waiting for CRDs to be installed
2023-04-18T22:49:00.432Z    FATAL    router/router.go:105    error waiting for CRDs    {"error": "timeout waiting for CRDs"}
```

With "default" as one of the resource namespaces, it works.

```
2023-04-18T23:06:22.365Z    DEBUG    utils/namespace.go:49    namespaces    {"function_namespace": "", "builder_namespace": "", "default_namespace": "", "fission_resource_namespace": ["default", "fission-datalake", "fission-ztps"]}
2023-04-18T23:06:22.375Z    INFO    otel/provider.go:50    OTEL_EXPORTER_OTLP_ENDPOINT not set, skipping Opentelemtry tracing
2023-04-18T23:06:22.376Z    INFO    crd/client.go:105    Waiting for CRDs to be installed
2023-04-18T23:06:22.390Z    INFO    router/router.go:214    starting router    {"port": 8888}
2023-04-18T23:06:22.390Z    INFO    httpserver/server.go:17    starting server    {"service": "router", "addr": ":8888"}
2023-04-18T23:06:22.390Z    INFO    httpserver/server.go:17    starting server    {"service": "metrics", "addr": ":8080"}   
```
I'm getting the same issue on v1.19.0

As @ytjohn wrote, adding default namespace to helm values helped, but now we have a bunch of resources (i.e roles) created on default namespace, which is against our internal policy to keep default clean/unused.

```
additionalFissionNamespaces:
  - default

```

Fix patch:
diff --git a/charts/fission-all/templates/_helpers.tpl b/charts/fission-all/templates/_helpers.tpl
index cbc63b2931..7bc43892c8 100644
--- a/charts/fission-all/templates/_helpers.tpl
+++ b/charts/fission-all/templates/_helpers.tpl
@@ -89,6 +89,12 @@ This template generates the image name for the deployment depending on the value
 {{- end }}
 
 {{- define "fission-resource-namespace.envs" }}
+- name: FISSION_BUILDER_NAMESPACE
+  value: "{{ .Values.builderNamespace }}"
+- name: FISSION_FUNCTION_NAMESPACE
+  value: "{{ .Values.functionNamespace }}"
+- name: FISSION_DEFAULT_NAMESPACE
+  value: "{{ .Values.defaultNamespace }}"
 - name: FISSION_RESOURCE_NAMESPACES
 {{- if gt (len .Values.additionalFissionNamespaces) 0 }}
   value: "{{ .Values.defaultNamespace }},{{ join "," .Values.additionalFissionNamespaces }}"
diff --git a/charts/fission-all/templates/buildermgr/deployment.yaml b/charts/fission-all/templates/buildermgr/deployment.yaml
index dd25609590..aaafc067c0 100644
--- a/charts/fission-all/templates/buildermgr/deployment.yaml
+++ b/charts/fission-all/templates/buildermgr/deployment.yaml
@@ -39,12 +39,6 @@ spec:
           value: "{{ .Values.pullPolicy }}"
         - name: BUILDER_IMAGE_PULL_POLICY
           value: "{{ .Values.pullPolicy }}"
-        - name: FISSION_BUILDER_NAMESPACE
-          value: "{{ .Values.builderNamespace }}"
-        - name: FISSION_FUNCTION_NAMESPACE
-          value: "{{ .Values.functionNamespace }}"
-        - name: FISSION_DEFAULT_NAMESPACE
-          value: "{{ .Values.defaultNamespace }}"
         - name: ENABLE_ISTIO
           value: "{{ .Values.enableIstio }}"
         - name: FETCHER_MINCPU
diff --git a/charts/fission-all/templates/controller/deployment.yaml b/charts/fission-all/templates/controller/deployment.yaml
index aad68e710c..51a19df696 100644
--- a/charts/fission-all/templates/controller/deployment.yaml
+++ b/charts/fission-all/templates/controller/deployment.yaml
@@ -33,12 +33,6 @@ spec:
         command: ["/fission-bundle"]
         args: ["--controllerPort", "8888"]
         env:
-        - name: FISSION_DEFAULT_NAMESPACE
-          value: "{{ .Values.defaultNamespace }}"
-        - name: FISSION_BUILDER_NAMESPACE
-          value: "{{ .Values.builderNamespace }}"
-        - name: FISSION_FUNCTION_NAMESPACE
-          value: "{{ .Values.functionNamespace }}"  
         - name: DEBUG_ENV
           value: {{ .Values.debugEnv | quote }}
         - name: PPROF_ENABLED
diff --git a/charts/fission-all/templates/executor/deployment.yaml b/charts/fission-all/templates/executor/deployment.yaml
index e626fa968a..6da414c29d 100644
--- a/charts/fission-all/templates/executor/deployment.yaml
+++ b/charts/fission-all/templates/executor/deployment.yaml
@@ -37,12 +37,6 @@ spec:
         {{- end }}
         - name: FETCHER_IMAGE_PULL_POLICY
           value: "{{ .Values.pullPolicy }}"
-        - name: FISSION_BUILDER_NAMESPACE
-          value: "{{ .Values.builderNamespace }}"
-        - name: FISSION_FUNCTION_NAMESPACE
-          value: "{{ .Values.functionNamespace }}"
-        - name: FISSION_DEFAULT_NAMESPACE
-          value: "{{ .Values.defaultNamespace }}"
         - name: RUNTIME_IMAGE_PULL_POLICY
           value: "{{ .Values.pullPolicy }}"
         - name: ADOPT_EXISTING_RESOURCES


Test patch: