Repository: cloudnativelabs/kube-router
Pull request #1650 (2024-04-21 17:01:39)

Fix Cleanup Code for v2.x
kube-router v2.X introduced the idea of iptables and ipset handlers that allow kube-router to be dual-stack capable. However, the cleanup logic for the various controllers was not properly ported when this happened. When the cleanup functions run, they often have not had their controllers fully initialized as cleanup should not be dependant on kube-router being able to reach a kube-apiserver.

As such, they were missing these handlers. And as such they either silently ended up doing noops or worse, they would run into nil pointer failures.

This corrects that, so that kube-router no longer fails this way and cleans up as it had in v1.X.

@vladimirtiukhtin - I tested this in my environment and found it worked correctly, but if you wouldn't mind checking as well, that would be helpful to ensure that there isn't something I'm missing. There should be a kube-router docker container that is built as part of this PR that you can find in the GitHub actions section if you don't want to build kube-router yourself.

Fixes: #1649 

vladimirtiukhtin: @aauren it works. Not without errors though
```
# kube-router --cleanup-config
I0422 08:55:24.774859       7 network_policy_controller.go:713] Cleaning up NetworkPolicyController configurations...
I0422 08:55:25.432990       7 network_policy_controller.go:766] Successfully cleaned the NetworkPolicyController configurations done by kube-router
I0422 08:55:25.433718       7 network_services_controller.go:1755] Cleaning up NetworkServiceController configurations...
WARN[0000] Running modprobe ip_vs failed with message: `modprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1 
I0422 08:55:25.437538       7 network_services_controller.go:1762] ipvs definitions don't have names associated with them for checking, during cleanup we assume that we own all of them and delete all ipvs definitions
I0422 08:55:25.446883       7 network_services_controller.go:1790] Processing IPv4 hairpin rule cleanup
I0422 08:55:25.449415       7 network_services_controller.go:1798] Processing IPv6 hairpin rule cleanup
E0422 08:55:25.470290       7 network_services_controller.go:566] Failed to run iptables command: running [/sbin/iptables -t filter -X KUBE-ROUTER-SERVICES --wait]: exit status 4: iptables v1.8.10 (nf_tables):  CHAIN_DEL failed (Resource busy): chain KUBE-ROUTER-SERVICES
I0422 08:55:26.553989       7 network_services_controller.go:1822] Successfully cleaned the NetworkServiceController configuration done by kube-router
I0422 08:55:26.554088       7 network_routes_controller.go:897] Cleaning up NetworkRoutesController configurations
I0422 08:55:26.561566       7 pod_egress.go:73] Deleted iptables rule to masquerade outbound traffic from pods.
W0422 08:55:27.626388       7 network_routes_controller.go:947] Error deleting ipset: ipset v7.19: Set cannot be destroyed: it is in use by a kernel component
W0422 08:55:27.646544       7 network_routes_controller.go:947] Error deleting ipset: ipset v7.19: The set with the given name does not exist
I0422 08:55:27.646589       7 network_routes_controller.go:951] Successfully cleaned the NetworkRoutesController configuration done by kube-router
```
aauren: So on my cluster, I was able to run without issues if I add in a bind mount for the `/lib/modules` directory (which is what the deployments in k8s do for kube-router) with the following:

```
sudo ctr run --privileged -t --mount type=bind,src=/lib/modules,dst=/lib/modules,options=rbind:ro --net-host docker.io/cloudnativelabs/kube-router-git:PR-1650@sha256:4109d544ae89cb08a694baf325bc54f15f449a3f36ae58bfe1be537a3b5f7e2d test /usr/local/bin/kube-router --cleanup-config
```

The ipset error are likely a symptom of the referencing iptables not being removed. Those errors are caused by this log line:

```
E0422 08:55:25.470290       7 network_services_controller.go:566] Failed to run iptables command: running [/sbin/iptables -t filter -X KUBE-ROUTER-SERVICES --wait]: exit status 4: iptables v1.8.10 (nf_tables):  CHAIN_DEL failed (Resource busy): chain KUBE-ROUTER-SERVICES
```

I can't reproduce that on my node, but that is probably due to not having the same contention over iptables as your cluster likely has. You mentioned in another post that you have configured some jobs to run with a frequency of 1 minute. If those types of nodes are the same ones where you are testing this, then you likely have a lot more going on with iptables than I do in my cluster.

My guess is that `--wait` isn't fully taking effect because the `xtables.lock` isn't being mounted into the container. I would try running with that mounted as well like the daemonset runs. So something like this:

```
sudo ctr run --privileged -t --mount type=bind,src=/lib/modules,dst=/lib/modules,options=rbind:ro --mount type=bind,src=/run/xtables.lock,dst=/run/xtables.lock,options=rbind:rw --net-host docker.io/cloudnativelabs/kube-router-git:PR-1650@sha256:4109d544ae89cb08a694baf325bc54f15f449a3f36ae58bfe1be537a3b5f7e2d test /usr/local/bin/kube-router --cleanup-config
```

See if that works for you?
aauren: I have updated the user-guide to show a better version of the cleanup command which should resolve any errors for most users.

Patch:
diff --git a/docs/user-guide.md b/docs/user-guide.md
index eac6c10f..ff0e1570 100644
--- a/docs/user-guide.md
+++ b/docs/user-guide.md
@@ -207,8 +207,23 @@ kube-router --master=http://192.168.1.99:8080/ --run-firewall=true --run-service
 Please delete kube-router daemonset and then clean up all the configurations done (to ipvs, iptables, ipset, ip routes
 etc) by kube-router on the node by running below command.
 
+### Docker
+
+```sh
+docker run --privileged --net=host \
+--mount type=bind,source=/lib/modules,target=/lib/modules,readonly \
+--mount type=bind,source=/run/xtables.lock,target=/run/xtables.lock,bind-propagation=rshared \
+cloudnativelabs/kube-router /usr/local/bin/kube-router --cleanup-config
+```
+
+### containerd
+
 ```sh
-docker run --privileged --net=host cloudnativelabs/kube-router --cleanup-config
+$ ctr image pull docker.io/cloudnativelabs/kube-router:latest
+$ ctr run --privileged -t --net-host \
+--mount type=bind,src=/lib/modules,dst=/lib/modules,options=rbind:ro \
+--mount type=bind,src=/run/xtables.lock,dst=/run/xtables.lock,options=rbind:rw \
+docker.io/cloudnativelabs/kube-router:latest kube-router-cleanup /usr/local/bin/kube-router --cleanup-config
 ```
 
 ## trying kube-router as alternative to kube-proxy
diff --git a/pkg/controllers/netpol/network_policy_controller.go b/pkg/controllers/netpol/network_policy_controller.go
index 57f44206..92f960f4 100644
--- a/pkg/controllers/netpol/network_policy_controller.go
+++ b/pkg/controllers/netpol/network_policy_controller.go
@@ -723,6 +723,26 @@ func (npc *NetworkPolicyController) cleanupStaleIPSets(activePolicyIPSets map[st
 func (npc *NetworkPolicyController) Cleanup() {
 	klog.Info("Cleaning up NetworkPolicyController configurations...")
 
+	if len(npc.iptablesCmdHandlers) < 1 {
+		iptablesCmdHandlers, ipSetHandlers, err := NewIPTablesHandlers(nil)
+		if err != nil {
+			klog.Errorf("unable to get iptables and ipset handlers: %v", err)
+			return
+		}
+		npc.iptablesCmdHandlers = iptablesCmdHandlers
+		npc.ipSetHandlers = ipSetHandlers
+
+		// Make other structures that we rely on
+		npc.iptablesSaveRestore = make(map[v1core.IPFamily]utils.IPTablesSaveRestorer, 2)
+		npc.iptablesSaveRestore[v1core.IPv4Protocol] = utils.NewIPTablesSaveRestore(v1core.IPv4Protocol)
+		npc.iptablesSaveRestore[v1core.IPv6Protocol] = utils.NewIPTablesSaveRestore(v1core.IPv6Protocol)
+		npc.filterTableRules = make(map[v1core.IPFamily]*bytes.Buffer, 2)
+		var buf bytes.Buffer
+		npc.filterTableRules[v1core.IPv4Protocol] = &buf
+		var buf2 bytes.Buffer
+		npc.filterTableRules[v1core.IPv6Protocol] = &buf2
+	}
+
 	var emptySet map[string]bool
 	// Take a dump (iptables-save) of the current filter table for cleanupStaleRules() to work on
 	for ipFamily, iptablesSaveRestore := range npc.iptablesSaveRestore {
@@ -762,7 +782,7 @@ func NewIPTablesHandlers(config *options.KubeRouterConfig) (
 	iptablesCmdHandlers := make(map[v1core.IPFamily]utils.IPTablesHandler, 2)
 	ipSetHandlers := make(map[v1core.IPFamily]utils.IPSetHandler, 2)
 
-	if config.EnableIPv4 {
+	if config == nil || config.EnableIPv4 {
 		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
 		if err != nil {
 			return nil, nil, fmt.Errorf("failed to create iptables handler: %w", err)
@@ -775,7 +795,7 @@ func NewIPTablesHandlers(config *options.KubeRouterConfig) (
 		}
 		ipSetHandlers[v1core.IPv4Protocol] = ipset
 	}
-	if config.EnableIPv6 {
+	if config == nil || config.EnableIPv6 {
 		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv6)
 		if err != nil {
 			return nil, nil, fmt.Errorf("failed to create iptables handler: %w", err)
diff --git a/pkg/controllers/proxy/hairpin_controller.go b/pkg/controllers/proxy/hairpin_controller.go
index 06bc372e..5801f5c3 100644
--- a/pkg/controllers/proxy/hairpin_controller.go
+++ b/pkg/controllers/proxy/hairpin_controller.go
@@ -75,15 +75,14 @@ func (hpc *hairpinController) ensureHairpinEnabledForPodInterface(endpointIP str
 		// WARN: This method is deprecated and will be removed once docker-shim is removed from kubelet.
 		pid, err = hpc.nsc.ln.getContainerPidWithDocker(containerID)
 		if err != nil {
-			return fmt.Errorf("failed to prepare endpoint %s to do direct server return due to %v",
-				endpointIP, err)
+			return fmt.Errorf("failed to get pod's (%s) pid for hairpinning due to %v", endpointIP, err)
 		}
 	} else {
 		// We expect CRI compliant runtimes here
 		// ugly workaround, refactoring of pkg/Proxy is required
 		pid, err = hpc.nsc.ln.getContainerPidWithCRI(hpc.nsc.dsr.runtimeEndpoint, containerID)
 		if err != nil {
-			return fmt.Errorf("failed to prepare endpoint %s to do DSR due to: %v", endpointIP, err)
+			return fmt.Errorf("failed to get pod's (%s) pid for hairpinning due to %v", endpointIP, err)
 		}
 	}
 	klog.V(2).Infof("Found PID %d for endpoint IP %s", pid, endpointIP)
diff --git a/pkg/controllers/proxy/network_services_controller.go b/pkg/controllers/proxy/network_services_controller.go
index 039c92d0..002b4acc 100644
--- a/pkg/controllers/proxy/network_services_controller.go
+++ b/pkg/controllers/proxy/network_services_controller.go
@@ -1782,6 +1782,16 @@ func (nsc *NetworkServicesController) Cleanup() {
 		handle.Close()
 	}
 
+	// In prep for further steps make sure that ipset and iptables handlers are created
+	if len(nsc.iptablesCmdHandlers) < 1 {
+		// Even though we have a config at this point (via passed param), we want to send nil so that the node will
+		// discover which IP address families it has and act accordingly
+		err = nsc.setupHandlers(nil, nil)
+		if err != nil {
+			klog.Errorf("could not cleanup because we couldn't create iptables/ipset command handlers due to: %v", err)
+		}
+	}
+
 	// cleanup iptables masquerade rule
 	err = nsc.deleteMasqueradeIptablesRule()
 	if err != nil {
@@ -1790,15 +1800,21 @@ func (nsc *NetworkServicesController) Cleanup() {
 	}
 
 	// cleanup iptables hairpin rules
-	err = nsc.deleteHairpinIptablesRules(v1.IPv4Protocol)
-	if err != nil {
-		klog.Errorf("Failed to cleanup iptables hairpin rules: %s", err.Error())
-		return
+	if _, ok := nsc.iptablesCmdHandlers[v1.IPv4Protocol]; ok {
+		klog.Info("Processing IPv4 hairpin rule cleanup")
+		err = nsc.deleteHairpinIptablesRules(v1.IPv4Protocol)
+		if err != nil {
+			klog.Errorf("Failed to cleanup iptables hairpin rules: %s", err.Error())
+			return
+		}
 	}
-	err = nsc.deleteHairpinIptablesRules(v1.IPv6Protocol)
-	if err != nil {
-		klog.Errorf("Failed to cleanup iptables hairpin rules: %s", err.Error())
-		return
+	if _, ok := nsc.iptablesCmdHandlers[v1.IPv6Protocol]; ok {
+		klog.Info("Processing IPv6 hairpin rule cleanup")
+		err = nsc.deleteHairpinIptablesRules(v1.IPv6Protocol)
+		if err != nil {
+			klog.Errorf("Failed to cleanup iptables hairpin rules: %s", err.Error())
+			return
+		}
 	}
 
 	nsc.cleanupIpvsFirewall()
@@ -1927,6 +1943,70 @@ func (nsc *NetworkServicesController) handleServiceDelete(obj interface{}) {
 	nsc.OnServiceUpdate(service)
 }
 
+// setupHandlers Here we test to see whether the node is IPv6 capable, if the user has enabled IPv6 (via command-line
+// options) and the node has an IPv6 address, the following method will return an IPv6 address
+func (nsc *NetworkServicesController) setupHandlers(config *options.KubeRouterConfig, node *v1.Node) error {
+	// node being nil covers the case where this function is called by something that doesn't have a kube-apiserver
+	// connection like the cleanup code. In this instance we want all possible iptables and ipset handlers
+	if node != nil {
+		nsc.nodeIPv4Addrs, nsc.nodeIPv6Addrs = utils.GetAllNodeIPs(node)
+	}
+
+	// We test for nil configs as the Cleanup() method often doesn't have a valid config in this respect, so rather
+	// than trying to guess options, it is better to just let the logic fallthrough. For the primary path to this func,
+	// NewNetworkServicesController, the config will not be nil and we want to check that we have options that match
+	// the node's capability to ensure sanity later down the road.
+	if config != nil {
+		if config.EnableIPv4 && len(nsc.nodeIPv4Addrs[v1.NodeInternalIP]) < 1 &&
+			len(nsc.nodeIPv4Addrs[v1.NodeExternalIP]) < 1 {
+			return fmt.Errorf("IPv4 was enabled, but no IPv4 address was found on the node")
+		}
+	}
+	nsc.isIPv4Capable = len(nsc.nodeIPv4Addrs) > 0
+	if config != nil {
+		if config.EnableIPv6 && len(nsc.nodeIPv6Addrs[v1.NodeInternalIP]) < 1 &&
+			len(nsc.nodeIPv6Addrs[v1.NodeExternalIP]) < 1 {
+			return fmt.Errorf("IPv6 was enabled, but no IPv6 address was found on the node")
+		}
+	}
+	nsc.isIPv6Capable = len(nsc.nodeIPv6Addrs) > 0
+
+	nsc.ipSetHandlers = make(map[v1.IPFamily]utils.IPSetHandler)
+	nsc.iptablesCmdHandlers = make(map[v1.IPFamily]utils.IPTablesHandler)
+	if node == nil || len(nsc.nodeIPv4Addrs) > 0 {
+		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv4 iptables handler: %v", err)
+			return fmt.Errorf("failed to create iptables handler: %w", err)
+		}
+		nsc.iptablesCmdHandlers[v1.IPv4Protocol] = iptHandler
+
+		ipset, err := utils.NewIPSet(false)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv4 ipset handler: %v", err)
+			return fmt.Errorf("failed to create ipset handler: %w", err)
+		}
+		nsc.ipSetHandlers[v1.IPv4Protocol] = ipset
+	}
+	if node == nil || len(nsc.nodeIPv6Addrs) > 0 {
+		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv6)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv6 iptables handler: %v", err)
+			return fmt.Errorf("failed to create iptables handler: %w", err)
+		}
+		nsc.iptablesCmdHandlers[v1.IPv6Protocol] = iptHandler
+
+		ipset, err := utils.NewIPSet(true)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv6 ipset handler: %v", err)
+			return fmt.Errorf("failed to create ipset handler: %w", err)
+		}
+		nsc.ipSetHandlers[v1.IPv6Protocol] = ipset
+	}
+
+	return nil
+}
+
 // NewNetworkServicesController returns NetworkServicesController object
 func NewNetworkServicesController(clientset kubernetes.Interface,
 	config *options.KubeRouterConfig, svcInformer cache.SharedIndexInformer,
@@ -2021,51 +2101,9 @@ func NewNetworkServicesController(clientset kubernetes.Interface,
 		return nil, err
 	}
 
-	// Here we test to see whether the node is IPv6 capable, if the user has enabled IPv6 (via command-line options)
-	// and the node has an IPv6 address, the following method will return an IPv6 address
-	nsc.nodeIPv4Addrs, nsc.nodeIPv6Addrs = utils.GetAllNodeIPs(node)
-	if config.EnableIPv4 && len(nsc.nodeIPv4Addrs[v1.NodeInternalIP]) < 1 &&
-		len(nsc.nodeIPv4Addrs[v1.NodeExternalIP]) < 1 {
-		return nil, fmt.Errorf("IPv4 was enabled, but no IPv4 address was found on the node")
-	}
-	nsc.isIPv4Capable = len(nsc.nodeIPv4Addrs) > 0
-	if config.EnableIPv6 && len(nsc.nodeIPv6Addrs[v1.NodeInternalIP]) < 1 &&
-		len(nsc.nodeIPv6Addrs[v1.NodeExternalIP]) < 1 {
-		return nil, fmt.Errorf("IPv6 was enabled, but no IPv6 address was found on the node")
-	}
-	nsc.isIPv6Capable = len(nsc.nodeIPv6Addrs) > 0
-
-	nsc.ipSetHandlers = make(map[v1.IPFamily]utils.IPSetHandler)
-	nsc.iptablesCmdHandlers = make(map[v1.IPFamily]utils.IPTablesHandler)
-	if len(nsc.nodeIPv4Addrs) > 0 {
-		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv4 iptables handler: %v", err)
-			return nil, fmt.Errorf("failed to create iptables handler: %w", err)
-		}
-		nsc.iptablesCmdHandlers[v1.IPv4Protocol] = iptHandler
-
-		ipset, err := utils.NewIPSet(false)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv4 ipset handler: %v", err)
-			return nil, fmt.Errorf("failed to create ipset handler: %w", err)
-		}
-		nsc.ipSetHandlers[v1.IPv4Protocol] = ipset
-	}
-	if len(nsc.nodeIPv6Addrs) > 0 {
-		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv6)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv6 iptables handler: %v", err)
-			return nil, fmt.Errorf("failed to create iptables handler: %w", err)
-		}
-		nsc.iptablesCmdHandlers[v1.IPv6Protocol] = iptHandler
-
-		ipset, err := utils.NewIPSet(true)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv6 ipset handler: %v", err)
-			return nil, fmt.Errorf("failed to create ipset handler: %w", err)
-		}
-		nsc.ipSetHandlers[v1.IPv6Protocol] = ipset
+	err = nsc.setupHandlers(config, node)
+	if err != nil {
+		return nil, err
 	}
 
 	automtu, err := utils.GetMTUFromNodeIP(nsc.primaryIP)
diff --git a/pkg/controllers/routing/network_routes_controller.go b/pkg/controllers/routing/network_routes_controller.go
index b347347a..8407cdae 100644
--- a/pkg/controllers/routing/network_routes_controller.go
+++ b/pkg/controllers/routing/network_routes_controller.go
@@ -896,6 +896,17 @@ func (nrc *NetworkRoutingController) setupOverlayTunnel(tunnelName string, nextH
 func (nrc *NetworkRoutingController) Cleanup() {
 	klog.Infof("Cleaning up NetworkRoutesController configurations")
 
+	// In prep for further steps make sure that ipset and iptables handlers are created
+	if len(nrc.iptablesCmdHandlers) < 1 {
+		// Even though we have a config at this point (via passed param), we want to send nil so that the node will
+		// discover which IP address families it has and act accordingly
+		err := nrc.setupHandlers(nil)
+		if err != nil {
+			klog.Errorf("could not cleanup because iptables/ipset handlers could not be created due to: %v", err)
+			return
+		}
+	}
+
 	// Pod egress cleanup
 	err := nrc.deletePodEgressRule()
 	if err != nil {
@@ -1359,7 +1370,55 @@ func (nrc *NetworkRoutingController) startBgpServer(grpcServer bool) error {
 	return nil
 }
 
-// func (nrc *NetworkRoutingController) getExternalNodeIPs(
+func (nrc *NetworkRoutingController) setupHandlers(node *v1core.Node) error {
+	var err error
+
+	// node being nil covers the case where this function is called by something that doesn't have a kube-apiserver
+	// connection like the cleanup code. In this instance we want all possible iptables and ipset handlers
+	if node != nil {
+		nrc.podIPv4CIDRs, nrc.podIPv6CIDRs, err = utils.GetPodCIDRsFromNodeSpecDualStack(node)
+		if err != nil {
+			klog.Fatalf("Failed to get pod CIDRs from node spec. kube-router relies on kube-controller-manager to"+
+				"allocate pod CIDRs for the node or an annotation `kube-router.io/pod-cidrs`. Error: %v", err)
+			return fmt.Errorf("failed to get pod CIDRs detail from Node.spec: %v", err)
+		}
+	}
+
+	nrc.iptablesCmdHandlers = make(map[v1core.IPFamily]utils.IPTablesHandler)
+	nrc.ipSetHandlers = make(map[v1core.IPFamily]utils.IPSetHandler)
+	if node == nil || len(nrc.nodeIPv4Addrs) > 0 {
+		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv4 iptables handler: %v", err)
+			return fmt.Errorf("failed to create iptables handler: %w", err)
+		}
+		nrc.iptablesCmdHandlers[v1core.IPv4Protocol] = iptHandler
+
+		ipset, err := utils.NewIPSet(false)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv4 ipset handler: %v", err)
+			return fmt.Errorf("failed to create ipset handler: %w", err)
+		}
+		nrc.ipSetHandlers[v1core.IPv4Protocol] = ipset
+	}
+	if node == nil || len(nrc.nodeIPv6Addrs) > 0 {
+		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv6)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv6 iptables handler: %v", err)
+			return fmt.Errorf("failed to create iptables handler: %w", err)
+		}
+		nrc.iptablesCmdHandlers[v1core.IPv6Protocol] = iptHandler
+
+		ipset, err := utils.NewIPSet(true)
+		if err != nil {
+			klog.Fatalf("Failed to allocate IPv6 ipset handler: %v", err)
+			return fmt.Errorf("failed to create ipset handler: %w", err)
+		}
+		nrc.ipSetHandlers[v1core.IPv6Protocol] = ipset
+	}
+
+	return nil
+}
 
 // NewNetworkRoutingController returns new NetworkRoutingController object
 func NewNetworkRoutingController(clientset kubernetes.Interface,
@@ -1473,44 +1532,9 @@ func NewNetworkRoutingController(clientset kubernetes.Interface,
 	}
 	nrc.podCidr = cidr
 
-	nrc.podIPv4CIDRs, nrc.podIPv6CIDRs, err = utils.GetPodCIDRsFromNodeSpecDualStack(node)
+	err = nrc.setupHandlers(node)
 	if err != nil {
-		klog.Fatalf("Failed to get pod CIDRs from node spec. kube-router relies on kube-controller-manager to"+
-			"allocate pod CIDRs for the node or an annotation `kube-router.io/pod-cidrs`. Error: %v", err)
-		return nil, fmt.Errorf("failed to get pod CIDRs detail from Node.spec: %v", err)
-	}
-
-	nrc.iptablesCmdHandlers = make(map[v1core.IPFamily]utils.IPTablesHandler)
-	nrc.ipSetHandlers = make(map[v1core.IPFamily]utils.IPSetHandler)
-	if len(nrc.nodeIPv4Addrs) > 0 {
-		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv4 iptables handler: %v", err)
-			return nil, fmt.Errorf("failed to create iptables handler: %w", err)
-		}
-		nrc.iptablesCmdHandlers[v1core.IPv4Protocol] = iptHandler
-
-		ipset, err := utils.NewIPSet(false)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv4 ipset handler: %v", err)
-			return nil, fmt.Errorf("failed to create ipset handler: %w", err)
-		}
-		nrc.ipSetHandlers[v1core.IPv4Protocol] = ipset
-	}
-	if len(nrc.nodeIPv6Addrs) > 0 {
-		iptHandler, err := iptables.NewWithProtocol(iptables.ProtocolIPv6)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv6 iptables handler: %v", err)
-			return nil, fmt.Errorf("failed to create iptables handler: %w", err)
-		}
-		nrc.iptablesCmdHandlers[v1core.IPv6Protocol] = iptHandler
-
-		ipset, err := utils.NewIPSet(true)
-		if err != nil {
-			klog.Fatalf("Failed to allocate IPv6 ipset handler: %v", err)
-			return nil, fmt.Errorf("failed to create ipset handler: %w", err)
-		}
-		nrc.ipSetHandlers[v1core.IPv6Protocol] = ipset
+		return nil, err
 	}
 
 	for _, handler := range nrc.ipSetHandlers {
diff --git a/pkg/utils/node.go b/pkg/utils/node.go
index 966a7170..3ce03153 100644
--- a/pkg/utils/node.go
+++ b/pkg/utils/node.go
@@ -17,31 +17,33 @@ import (
 
 // GetNodeObject returns the node API object for the node
 func GetNodeObject(clientset kubernetes.Interface, hostnameOverride string) (*apiv1.Node, error) {
+	// if env NODE_NAME is not set and node is not registered with hostname, then use host name override
+	if hostnameOverride != "" {
+		node, err := clientset.CoreV1().Nodes().Get(context.Background(), hostnameOverride, metav1.GetOptions{})
+		if err != nil {
+			return nil, fmt.Errorf("unable to get node %s, due to: %v", hostnameOverride, err)
+		}
+		return node, nil
+	}
+
 	// assuming kube-router is running as pod, first check env NODE_NAME
 	nodeName := os.Getenv("NODE_NAME")
 	if nodeName != "" {
 		node, err := clientset.CoreV1().Nodes().Get(context.Background(), nodeName, metav1.GetOptions{})
-		if err == nil {
-			return node, nil
+		if err != nil {
+			return nil, fmt.Errorf("unable to get node %s, due to: %v", nodeName, err)
 		}
+		return node, nil
 	}
 
 	// if env NODE_NAME is not set then check if node is register with hostname
 	hostName, _ := os.Hostname()
 	node, err := clientset.CoreV1().Nodes().Get(context.Background(), hostName, metav1.GetOptions{})
-	if err == nil {
-		return node, nil
-	}
-
-	// if env NODE_NAME is not set and node is not registered with hostname, then use host name override
-	if hostnameOverride != "" {
-		node, err = clientset.CoreV1().Nodes().Get(context.Background(), hostnameOverride, metav1.GetOptions{})
-		if err == nil {
-			return node, nil
-		}
+	if err != nil {
+		return nil, fmt.Errorf("failed to identify the node by NODE_NAME, %s or --hostname-override: %v", hostName, err)
 	}
 
-	return nil, fmt.Errorf("failed to identify the node by NODE_NAME, hostname or --hostname-override")
+	return node, nil
 }
 
 // GetPrimaryNodeIP returns the most valid external facing IP address for a node.


Test patch:
diff --git a/pkg/controllers/netpol/network_policy_controller_test.go b/pkg/controllers/netpol/network_policy_controller_test.go
index fd3ef84e..f20faafb 100644
--- a/pkg/controllers/netpol/network_policy_controller_test.go
+++ b/pkg/controllers/netpol/network_policy_controller_test.go
@@ -5,6 +5,7 @@ import (
 	"context"
 	"fmt"
 	"net"
+	"os"
 	"strings"
 	"sync"
 	"testing"
@@ -754,6 +755,7 @@ func (ips *fakeIPSet) Name(name string) string {
 }
 
 func TestNetworkPolicyController(t *testing.T) {
+	curHostname, _ := os.Hostname()
 	testCases := []tNetPolConfigTestCase{
 		{
 			"Default options are successful",
@@ -765,7 +767,8 @@ func TestNetworkPolicyController(t *testing.T) {
 			"Missing nodename fails appropriately",
 			newMinimalKubeRouterConfig([]string{""}, "", "", nil, nil, false),
 			true,
-			"failed to identify the node by NODE_NAME, hostname or --hostname-override",
+			fmt.Sprintf("failed to identify the node by NODE_NAME, %s or --hostname-override: nodes \"%s\""+
+				" not found", curHostname, curHostname),
 		},
 		{
 			"Test bad cluster CIDR (not properly formatting ip address)",
diff --git a/pkg/utils/node_test.go b/pkg/utils/node_test.go
index fff1419e..bc464ba0 100644
--- a/pkg/utils/node_test.go
+++ b/pkg/utils/node_test.go
@@ -38,7 +38,7 @@ func Test_GetNodeObject(t *testing.T) {
 			nil,
 		},
 		{
-			"node with hostname overrie exists",
+			"node with hostname override exists",
 			"something-else",
 			"test-node",
 			&apiv1.Node{
@@ -50,8 +50,8 @@ func Test_GetNodeObject(t *testing.T) {
 		},
 		{
 			"node with current hostname exists",
-			"something-else",
-			"something-else",
+			"",
+			"",
 			&apiv1.Node{
 				ObjectMeta: metav1.ObjectMeta{
 					Name: curHostname,
@@ -68,7 +68,7 @@ func Test_GetNodeObject(t *testing.T) {
 					Name: "another-node",
 				},
 			},
-			errors.New("failed to identify the node by NODE_NAME, hostname or --hostname-override"),
+			errors.New("unable to get node test-node, due to: nodes \"test-node\" not found"),
 		},
 	}