Repository: cnoe-io/idpbuilder
Pull request #495: fix bugs around ingress, tls, and dev-password
Author: login: squidboylan
Created: 2025-03-07T00:39:13Z | Merged: 2025-03-12T22:43:20Z

This PR fixes a few bugs around using the `--ingress-host-name` flag, `--dev-password` flag, and `--use-path-routing` flag. I initially wanted to separate these but decided they make sense to keep together since they're all a little bit interdependent.

Before this change using the `--ingress-host-name` flag would cause argo to fail to fetch repos from gitea due to missing SANs in the TLS cert. It also fixes a bug where we're hardcoding the domain for argocd and gitea to `cnoe.localtest.me` in the `--dev-password` codepath.


Diff:
diff --git a/hack/argo-cd/ingress.yaml.tmpl b/hack/argo-cd/ingress.yaml.tmpl
index 30149da30..26256db6b 100644
--- a/hack/argo-cd/ingress.yaml.tmpl
+++ b/hack/argo-cd/ingress.yaml.tmpl
@@ -22,7 +22,8 @@ spec:
                 name: argocd-server
                 port:
                   name: http
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - path: /argocd(/|$)(.*)
@@ -32,7 +33,7 @@ spec:
                 name: argocd-server
                 port:
                   name: http
-
+{{ end }}
 {{- else -}}
 ---
 apiVersion: networking.k8s.io/v1
@@ -56,4 +57,16 @@ spec:
                 name: argocd-server
                 port:
                   name: https
+{{- if ne .IngressHost .Host }}
+    - host: argocd.{{ .Host }}
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: argocd-server
+                port:
+                  name: https
+{{ end }}
 {{ end }}
diff --git a/hack/gitea/ingress.yaml.tmpl b/hack/gitea/ingress.yaml.tmpl
index 4f19d92d3..2226345a9 100644
--- a/hack/gitea/ingress.yaml.tmpl
+++ b/hack/gitea/ingress.yaml.tmpl
@@ -20,7 +20,8 @@ spec:
                   number: 3000
             path: /v2
             pathType: Prefix
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - backend:
@@ -30,6 +31,7 @@ spec:
                   number: 3000
             path: /v2
             pathType: Prefix
+{{ end }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -53,7 +55,8 @@ spec:
                   number: 3000
             path: /v2/gitea(/|$)(.*)
             pathType: ImplementationSpecific
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - backend:
@@ -63,6 +66,7 @@ spec:
                   number: 3000
             path: /v2/gitea(/|$)(.*)
             pathType: ImplementationSpecific
+{{ end }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -86,7 +90,8 @@ spec:
                   number: 3000
             path: /gitea(/|$)(.*)
             pathType: ImplementationSpecific
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - backend:
@@ -96,7 +101,8 @@ spec:
                   number: 3000
             path: /gitea(/|$)(.*)
             pathType: ImplementationSpecific
-{{ else if ( ne .Host "cnoe.localtest.me") }}
+{{ end }}
+{{ else }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -118,18 +124,8 @@ spec:
                 name: my-gitea-http
                 port:
                   number: 3000
-{{ end }}
----
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: my-gitea
-  annotations:
-    nginx.ingress.kubernetes.io/proxy-body-size: 1024m
-spec:
-  ingressClassName: nginx
-  rules:
-    - host: "gitea.cnoe.localtest.me"
+{{- if ne .IngressHost .Host }}
+    - host: gitea.{{ .Host }}
       http:
         paths:
           - path: /
@@ -139,3 +135,5 @@ spec:
                 name: my-gitea-http
                 port:
                   number: 3000
+{{ end }}
+{{ end }}
diff --git a/pkg/build/tls.go b/pkg/build/tls.go
index e975598cb..dc93977e7 100644
--- a/pkg/build/tls.go
+++ b/pkg/build/tls.go
@@ -201,6 +201,9 @@ func setupSelfSignedCertificate(ctx context.Context, logger logr.Logger, kubecli
 			fmt.Sprintf("*.%s", config.Host),
 		}
 	}
+	if config.IngressHost != config.Host {
+		sans = append(sans, config.IngressHost, fmt.Sprintf("*.%s", config.IngressHost))
+	}
 
 	logger.V(1).Info("Creating/getting certificate", "host", config.Host, "sans", sans)
 	cert, privateKey, err := getOrCreateIngressCertificateAndKey(ctx, kubeclient, globals.SelfSignedCertSecretName, globals.NginxNamespace, sans)
diff --git a/pkg/controllers/localbuild/gitea.go b/pkg/controllers/localbuild/gitea.go
index b83c2de94..ae34d6108 100644
--- a/pkg/controllers/localbuild/gitea.go
+++ b/pkg/controllers/localbuild/gitea.go
@@ -157,7 +157,7 @@ func (r *LocalbuildReconciler) setGiteaToken(ctx context.Context, secret corev1.
 // gitea URL reachable within the cluster with proper coredns config. Mainly for argocd
 func giteaInternalBaseUrl(config v1alpha1.BuildCustomizationSpec) string {
 	if config.UsePathRouting {
-		return fmt.Sprintf(util.GiteaSvcURL, config.Protocol, "", config.Host, config.Port, "/gitea")
+		return fmt.Sprintf(util.GiteaURLTempl, config.Protocol, "", config.Host, config.Port, "/gitea")
 	}
-	return fmt.Sprintf(util.GiteaSvcURL, config.Protocol, "gitea.", config.Host, config.Port, "")
+	return fmt.Sprintf(util.GiteaURLTempl, config.Protocol, "gitea.", config.Host, config.Port, "")
 }
diff --git a/pkg/controllers/localbuild/resources/argo/ingress.yaml b/pkg/controllers/localbuild/resources/argo/ingress.yaml
index 30149da30..26256db6b 100644
--- a/pkg/controllers/localbuild/resources/argo/ingress.yaml
+++ b/pkg/controllers/localbuild/resources/argo/ingress.yaml
@@ -22,7 +22,8 @@ spec:
                 name: argocd-server
                 port:
                   name: http
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - path: /argocd(/|$)(.*)
@@ -32,7 +33,7 @@ spec:
                 name: argocd-server
                 port:
                   name: http
-
+{{ end }}
 {{- else -}}
 ---
 apiVersion: networking.k8s.io/v1
@@ -56,4 +57,16 @@ spec:
                 name: argocd-server
                 port:
                   name: https
+{{- if ne .IngressHost .Host }}
+    - host: argocd.{{ .Host }}
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: argocd-server
+                port:
+                  name: https
+{{ end }}
 {{ end }}
diff --git a/pkg/controllers/localbuild/resources/gitea/k8s/install.yaml b/pkg/controllers/localbuild/resources/gitea/k8s/install.yaml
index 649d1eeaa..1438cf383 100644
--- a/pkg/controllers/localbuild/resources/gitea/k8s/install.yaml
+++ b/pkg/controllers/localbuild/resources/gitea/k8s/install.yaml
@@ -649,7 +649,8 @@ spec:
                   number: 3000
             path: /v2
             pathType: Prefix
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - backend:
@@ -659,6 +660,7 @@ spec:
                   number: 3000
             path: /v2
             pathType: Prefix
+{{ end }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -682,7 +684,8 @@ spec:
                   number: 3000
             path: /v2/gitea(/|$)(.*)
             pathType: ImplementationSpecific
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - backend:
@@ -692,6 +695,7 @@ spec:
                   number: 3000
             path: /v2/gitea(/|$)(.*)
             pathType: ImplementationSpecific
+{{ end }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -715,7 +719,8 @@ spec:
                   number: 3000
             path: /gitea(/|$)(.*)
             pathType: ImplementationSpecific
-    - host: localhost
+{{- if ne .IngressHost .Host }}
+    - host: {{ .Host }}
       http:
         paths:
           - backend:
@@ -725,7 +730,8 @@ spec:
                   number: 3000
             path: /gitea(/|$)(.*)
             pathType: ImplementationSpecific
-{{ else if ( ne .Host "cnoe.localtest.me") }}
+{{ end }}
+{{ else }}
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -747,18 +753,8 @@ spec:
                 name: my-gitea-http
                 port:
                   number: 3000
-{{ end }}
----
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: my-gitea
-  annotations:
-    nginx.ingress.kubernetes.io/proxy-body-size: 1024m
-spec:
-  ingressClassName: nginx
-  rules:
-    - host: "gitea.cnoe.localtest.me"
+{{- if ne .IngressHost .Host }}
+    - host: gitea.{{ .Host }}
       http:
         paths:
           - path: /
@@ -768,3 +764,5 @@ spec:
                 name: my-gitea-http
                 port:
                   number: 3000
+{{ end }}
+{{ end }}
diff --git a/pkg/util/argocd.go b/pkg/util/argocd.go
index 34477613d..dfdadfdd4 100644
--- a/pkg/util/argocd.go
+++ b/pkg/util/argocd.go
@@ -12,7 +12,7 @@ const (
 	ArgocdInitialAdminSecretName = "argocd-initial-admin-secret"
 	ArgocdAdminName              = "admin"
 	ArgocdNamespace              = "argocd"
-	ArgocdIngressURL             = "%s://argocd.cnoe.localtest.me:%s"
+	ArgocdURLTempl               = "%s://%s%s:%s%s"
 )
 
 func ArgocdBaseUrl(ctx context.Context) (string, error) {
@@ -20,7 +20,10 @@ func ArgocdBaseUrl(ctx context.Context) (string, error) {
 	if err != nil {
 		return "", fmt.Errorf("error fetching idp config: %s", err)
 	}
-	return fmt.Sprintf(ArgocdIngressURL, idpConfig.Protocol, idpConfig.Port), nil
+	if idpConfig.UsePathRouting {
+		return fmt.Sprintf(ArgocdURLTempl, idpConfig.Protocol, "", idpConfig.Host, idpConfig.Port, "/argocd"), nil
+	}
+	return fmt.Sprintf(ArgocdURLTempl, idpConfig.Protocol, "argocd.", idpConfig.Host, idpConfig.Port, ""), nil
 }
 
 func ArgocdInitialAdminSecretObject() corev1.Secret {
diff --git a/pkg/util/gitea.go b/pkg/util/gitea.go
index 62415c23c..fd728dacc 100644
--- a/pkg/util/gitea.go
+++ b/pkg/util/gitea.go
@@ -21,11 +21,7 @@ const (
 	GiteaAdminName           = "giteaAdmin"
 	GiteaAdminTokenName      = "admin"
 	GiteaAdminTokenFieldName = "token"
-	// this is the URL accessible outside cluster. resolves to localhost
-	GiteaIngressURL = "%s://gitea.cnoe.localtest.me:%s"
-	// this is the URL accessible within cluster for ArgoCD to fetch resources.
-	// resolves to cluster ip
-	GiteaSvcURL = "%s://%s%s:%s%s"
+	GiteaURLTempl            = "%s://%s%s:%s%s"
 )
 
 func GiteaAdminSecretObject() corev1.Secret {
@@ -118,5 +114,8 @@ func GiteaBaseUrl(ctx context.Context) (string, error) {
 	if err != nil {
 		return "", fmt.Errorf("error fetching idp config: %s", err)
 	}
-	return fmt.Sprintf(GiteaIngressURL, idpConfig.Protocol, idpConfig.Port), nil
+	if idpConfig.UsePathRouting {
+		return fmt.Sprintf(GiteaURLTempl, idpConfig.Protocol, "", idpConfig.Host, idpConfig.Port, "/gitea"), nil
+	}
+	return fmt.Sprintf(GiteaURLTempl, idpConfig.Protocol, "gitea.", idpConfig.Host, idpConfig.Port, ""), nil
 }


Reviews:
user: login: squidboylan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T04:27:34Z
user: login: punkwalker
body: The changes LGTM. I am not sure how/if these changes will impact the current assumptions that we have made in configuration of various components. Let's see how e2e test perform on this PR.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T18:05:22Z
user: login: punkwalker
body: LGTM 🚀 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T22:42:47Z

Review comments:
diff_hunk: @@ -157,7 +157,7 @@ func (r *LocalbuildReconciler) setGiteaToken(ctx context.Context, secret corev1.
 // gitea URL reachable within the cluster with proper coredns config. Mainly for argocd
 func giteaInternalBaseUrl(config v1alpha1.BuildCustomizationSpec) string {
path: pkg/controllers/localbuild/gitea.go
user: login: squidboylan
body: This could be simplified a bit to just use the pkg/util/gitea.go `GiteaBaseUrl` function, not sure if we want that change in this PR though, it's already getting a bit bloated
created_at: 2025-03-07T04:27:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -157,7 +157,7 @@ func (r *LocalbuildReconciler) setGiteaToken(ctx context.Context, secret corev1.
 // gitea URL reachable within the cluster with proper coredns config. Mainly for argocd
 func giteaInternalBaseUrl(config v1alpha1.BuildCustomizationSpec) string {
path: pkg/controllers/localbuild/gitea.go
user: login: punkwalker
body: I think this function is specific for returning in-cluster URL and I can see it is only used once. And the way this PR changes it's logic, it is identical to `pkg/util/gitea.go/GiteaBaseUrl` function. So we should use that and get rid of this function.
created_at: 2025-03-07T18:02:49Z
author_association: CONTRIBUTOR

PR comments:
user: login: squidboylan
created_at: 2025-03-07T00:53:49Z
body: Marking this as a draft, i think there's more bugs here and i wanna make sure I'm solving them right
author_association: CONTRIBUTOR
user: login: punkwalker
created_at: 2025-03-07T18:05:44Z
body: /e2e
author_association: CONTRIBUTOR
user: login: punkwalker
created_at: 2025-03-07T19:08:52Z
body: > The changes LGTM. I am not sure how/if these changes will impact the current assumptions that we have made in configuration of various components. Let's see how e2e test perform on this PR.

So e2e completed successfully. So we can merge this if you are done with changes
author_association: CONTRIBUTOR
user: login: squidboylan
created_at: 2025-03-07T20:43:53Z
body: > So e2e completed successfully. So we can merge this if you are done with changes

Let's merge this, I'll remove the duplicate function in a future PR, this PR is already getting larger than it should be.
author_association: CONTRIBUTOR