Repository: kubernetes-sigs/kwok
Pull request #1351: Add enablePodsOnNodeSyncListPager option to switch between watch list and pager list to sync pods on node
Author: login: lianghao208
Labels: name: kind/feature
description: Categorizes issue or PR as related to a new feature.
name: approved
description: Indicates a PR has been approved by an approver from all required OWNERS files.
name: kind/documentation
description: Categorizes issue or PR as related to documentation.
name: kind/api-change
description: Categorizes issue or PR as related to adding, removing, or otherwise changing an API
name: cncf-cla: yes
description: Indicates the PR's author has signed the CNCF CLA.
name: release-note
description: Denotes a PR that will be considered when it comes time to generate release notes.
name: lgtm
description: "Looks good to me", indicates that a PR is ready to be merged.
name: size/M
description: Denotes a PR that changes 30-99 lines, ignoring generated files.
Created: 2025-04-21T03:32:26Z | Merged: 2025-04-23T06:01:41Z

<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind feature
<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
Add enablePodsOnNodeSyncListPager option to switch between watch list and pager list to sync pods on node.
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```



Diff:
diff --git a/pkg/apis/config/v1alpha1/kwok_configuration_types.go b/pkg/apis/config/v1alpha1/kwok_configuration_types.go
index e6b64fde5d..d052ce4aed 100644
--- a/pkg/apis/config/v1alpha1/kwok_configuration_types.go
+++ b/pkg/apis/config/v1alpha1/kwok_configuration_types.go
@@ -153,6 +153,10 @@ type KwokConfigurationOptions struct {
 	// PodsOnNodeSyncParallelism is the number of workers to sync pods on nodes in parallel.
 	// +default=1
 	PodsOnNodeSyncParallelism uint `json:"podsOnNodeSyncParallelism,omitempty"`
+
+	// EnablePodsOnNodeSyncListPager enables pager list for workers to sync pods on nodes.
+	// +default=true
+	EnablePodsOnNodeSyncListPager *bool `json:"enablePodsOnNodeSyncListPager"`
 }
 
 // TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.
diff --git a/pkg/apis/config/v1alpha1/zz_generated.deepcopy.go b/pkg/apis/config/v1alpha1/zz_generated.deepcopy.go
index 7b170aaf8e..556a2de557 100644
--- a/pkg/apis/config/v1alpha1/zz_generated.deepcopy.go
+++ b/pkg/apis/config/v1alpha1/zz_generated.deepcopy.go
@@ -227,6 +227,11 @@ func (in *KwokConfigurationOptions) DeepCopyInto(out *KwokConfigurationOptions)
 		*out = new(bool)
 		**out = **in
 	}
+	if in.EnablePodsOnNodeSyncListPager != nil {
+		in, out := &in.EnablePodsOnNodeSyncListPager, &out.EnablePodsOnNodeSyncListPager
+		*out = new(bool)
+		**out = **in
+	}
 	return
 }
 
diff --git a/pkg/apis/config/v1alpha1/zz_generated.defaults.go b/pkg/apis/config/v1alpha1/zz_generated.defaults.go
index 45b676a98e..3cd8c7db99 100644
--- a/pkg/apis/config/v1alpha1/zz_generated.defaults.go
+++ b/pkg/apis/config/v1alpha1/zz_generated.defaults.go
@@ -70,6 +70,10 @@ func SetObjectDefaults_KwokConfiguration(in *KwokConfiguration) {
 	if in.Options.PodsOnNodeSyncParallelism == 0 {
 		in.Options.PodsOnNodeSyncParallelism = 1
 	}
+	if in.Options.EnablePodsOnNodeSyncListPager == nil {
+		var ptrVar1 bool = true
+		in.Options.EnablePodsOnNodeSyncListPager = &ptrVar1
+	}
 }
 
 func SetObjectDefaults_KwokctlConfiguration(in *KwokctlConfiguration) {
diff --git a/pkg/apis/internalversion/kwok_configuration_types.go b/pkg/apis/internalversion/kwok_configuration_types.go
index d201835c9b..423f408153 100644
--- a/pkg/apis/internalversion/kwok_configuration_types.go
+++ b/pkg/apis/internalversion/kwok_configuration_types.go
@@ -105,6 +105,9 @@ type KwokConfigurationOptions struct {
 
 	// PodsOnNodeSyncParallelism is the number of workers to sync pods on nodes in parallel.
 	PodsOnNodeSyncParallelism uint
+
+	// EnablePodsOnNodeSyncListPager enables pager list for workers to sync pods on nodes.
+	EnablePodsOnNodeSyncListPager bool
 }
 
 // TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.
diff --git a/pkg/apis/internalversion/zz_generated.conversion.go b/pkg/apis/internalversion/zz_generated.conversion.go
index e8cc4dd171..bbae9ac355 100644
--- a/pkg/apis/internalversion/zz_generated.conversion.go
+++ b/pkg/apis/internalversion/zz_generated.conversion.go
@@ -1534,6 +1534,9 @@ func autoConvert_internalversion_KwokConfigurationOptions_To_v1alpha1_KwokConfig
 	out.NodeLeaseDurationSeconds = in.NodeLeaseDurationSeconds
 	out.NodeLeaseParallelism = in.NodeLeaseParallelism
 	out.PodsOnNodeSyncParallelism = in.PodsOnNodeSyncParallelism
+	if err := v1.Convert_bool_To_Pointer_bool(&in.EnablePodsOnNodeSyncListPager, &out.EnablePodsOnNodeSyncListPager, s); err != nil {
+		return err
+	}
 	return nil
 }
 
@@ -1576,6 +1579,9 @@ func autoConvert_v1alpha1_KwokConfigurationOptions_To_internalversion_KwokConfig
 	out.NodeLeaseDurationSeconds = in.NodeLeaseDurationSeconds
 	out.NodeLeaseParallelism = in.NodeLeaseParallelism
 	out.PodsOnNodeSyncParallelism = in.PodsOnNodeSyncParallelism
+	if err := v1.Convert_Pointer_bool_To_bool(&in.EnablePodsOnNodeSyncListPager, &out.EnablePodsOnNodeSyncListPager, s); err != nil {
+		return err
+	}
 	return nil
 }
 
diff --git a/pkg/kwok/cmd/root.go b/pkg/kwok/cmd/root.go
index 8e90512a68..469fcbe1e9 100644
--- a/pkg/kwok/cmd/root.go
+++ b/pkg/kwok/cmd/root.go
@@ -293,6 +293,7 @@ func runE(ctx context.Context, flags *flagpole) error {
 		NodeLeaseDurationSeconds:              flags.Options.NodeLeaseDurationSeconds,
 		ID:                                    id,
 		PodsOnNodeSyncParallelism:             flags.Options.PodsOnNodeSyncParallelism,
+		EnablePodsOnNodeSyncListPager:         flags.Options.EnablePodsOnNodeSyncListPager,
 	})
 	if err != nil {
 		return err
diff --git a/pkg/kwok/controllers/controller.go b/pkg/kwok/controllers/controller.go
index e5e39d6741..caa3b24d59 100644
--- a/pkg/kwok/controllers/controller.go
+++ b/pkg/kwok/controllers/controller.go
@@ -123,6 +123,7 @@ type Config struct {
 	NodeLeaseDurationSeconds              uint
 	NodeLeaseParallelism                  uint
 	PodsOnNodeSyncParallelism             uint
+	EnablePodsOnNodeSyncListPager         bool
 	ID                                    string
 	EnableMetrics                         bool
 	EnablePodCache                        bool
@@ -566,9 +567,13 @@ func (c *Controller) podsOnNodeSyncWorker(ctx context.Context) {
 		if !ok {
 			return
 		}
-		err := c.podsInformer.Sync(ctx, informer.Option{
-			FieldSelector: fields.OneTermEqualSelector("spec.nodeName", nodeName).String(),
-		}, c.podsChan)
+
+		opt := informer.Option{
+			FieldSelector:   fields.OneTermEqualSelector("spec.nodeName", nodeName).String(),
+			EnableListPager: c.conf.EnablePodsOnNodeSyncListPager,
+		}
+
+		err := c.podsInformer.Sync(ctx, opt, c.podsChan)
 		if err != nil {
 			logger.Error("failed to update pods on node", err, "node", nodeName)
 		}
diff --git a/pkg/kwok/controllers/controller_test.go b/pkg/kwok/controllers/controller_test.go
index 805bb0e808..af7634edda 100644
--- a/pkg/kwok/controllers/controller_test.go
+++ b/pkg/kwok/controllers/controller_test.go
@@ -172,6 +172,28 @@ func TestController(t *testing.T) {
 			},
 			wantErr: false,
 		},
+		{
+			name: "node controller test: manage all nodes by watch list",
+			conf: Config{
+				TypedClient:    fake.NewSimpleClientset(nodes...),
+				ManageAllNodes: true,
+				LocalStages: map[internalversion.StageResourceRef][]*internalversion.Stage{
+					podRef:  podStages,
+					nodeRef: nodeStages,
+				},
+				CIDR:                          "10.0.0.1/24",
+				NodePlayStageParallelism:      1,
+				PodPlayStageParallelism:       1,
+				PodsOnNodeSyncParallelism:     1,
+				EnablePodsOnNodeSyncListPager: false,
+			},
+			wantNodePhase: map[string]corev1.NodePhase{
+				"node-0": corev1.NodeRunning,
+				"node-1": corev1.NodeRunning,
+				"node-2": corev1.NodeRunning,
+			},
+			wantErr: false,
+		},
 	}
 
 	ctx := context.Background()
diff --git a/pkg/utils/informer/event.go b/pkg/utils/informer/event.go
index 6a14a9fc06..8426563b11 100644
--- a/pkg/utils/informer/event.go
+++ b/pkg/utils/informer/event.go
@@ -61,6 +61,7 @@ type Option struct {
 	LabelSelector      string
 	FieldSelector      string
 	AnnotationSelector string
+	EnableListPager    bool
 	annotationSelector labels.Selector
 }
 
@@ -71,6 +72,9 @@ func (o *Option) setup(opts *metav1.ListOptions) {
 	if o.FieldSelector != "" {
 		opts.FieldSelector = o.FieldSelector
 	}
+	if !o.EnableListPager {
+		opts.ResourceVersion = "0"
+	}
 }
 
 func (o *Option) toListOptions() metav1.ListOptions {
diff --git a/pkg/utils/informer/informer.go b/pkg/utils/informer/informer.go
index db43b9b3ab..588a073155 100644
--- a/pkg/utils/informer/informer.go
+++ b/pkg/utils/informer/informer.go
@@ -21,6 +21,7 @@ import (
 	"fmt"
 	"sync"
 
+	"k8s.io/apimachinery/pkg/api/meta"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/runtime"
@@ -48,11 +49,8 @@ func (i *Informer[T, L]) Sync(ctx context.Context, opt Option, events chan<- Eve
 	if events == nil {
 		return fmt.Errorf("events channel is nil")
 	}
-	listPager := pager.New(func(ctx context.Context, opts metav1.ListOptions) (runtime.Object, error) {
-		return i.ListFunc(ctx, opts)
-	})
 
-	err := listPager.EachListItem(ctx, opt.toListOptions(), func(obj runtime.Object) error {
+	fun := func(obj runtime.Object) error {
 		if ok, err := opt.filter(obj); err != nil {
 			return err
 		} else if !ok {
@@ -60,7 +58,20 @@ func (i *Informer[T, L]) Sync(ctx context.Context, opt Option, events chan<- Eve
 		}
 		events <- Event[T]{Type: Sync, Object: obj.(T)}
 		return nil
+	}
+	if !opt.EnableListPager {
+		list, err := i.ListFunc(ctx, opt.toListOptions())
+		if err != nil {
+			return err
+		}
+		return meta.EachListItem(list, fun)
+	}
+
+	listPager := pager.New(func(ctx context.Context, opts metav1.ListOptions) (runtime.Object, error) {
+		return i.ListFunc(ctx, opts)
 	})
+
+	err := listPager.EachListItem(ctx, opt.toListOptions(), fun)
 	if err != nil {
 		return err
 	}
diff --git a/site/content/en/docs/generated/apis.md b/site/content/en/docs/generated/apis.md
index e9b5456f77..2b830e576d 100644
--- a/site/content/en/docs/generated/apis.md
+++ b/site/content/en/docs/generated/apis.md
@@ -2658,6 +2658,17 @@ uint
 <p>PodsOnNodeSyncParallelism is the number of workers to sync pods on nodes in parallel.</p>
 </td>
 </tr>
+<tr>
+<td>
+<code>enablePodsOnNodeSyncListPager</code>
+<em>
+bool
+</em>
+</td>
+<td>
+<p>EnablePodsOnNodeSyncListPager enables pager list for workers to sync pods on nodes.</p>
+</td>
+</tr>
 </tbody>
 </table>
 <h3 id="config.kwok.x-k8s.io/v1alpha1.KwokctlConfigurationOptions">


Reviews:
user: login: wzshiming
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T09:28:35Z

Review comments:
diff_hunk: @@ -48,6 +48,13 @@ func (i *Informer[T, L]) Sync(ctx context.Context, opt Option, events chan<- Eve
 	if events == nil {
 		return fmt.Errorf("events channel is nil")
 	}
+
+	if opt.EnableWatchList {
+		informer := newDummyInformerWithWatchList(i.listWatch(ctx), opt, events)
+		go informer.Run(ctx.Done())
+		return nil
+	}
+
path: pkg/utils/informer/informer.go
user: login: wzshiming
body: ```suggestion
	fun := func(obj runtime.Object) error {
		if ok, err := opt.filter(obj); err != nil {
			return err
		} else if !ok {
			return nil
		}
		events <- Event[T]{Type: Sync, Object: obj.(T)}
		return nil
	}

	if opt.EnableWatchList {
		list, err := i.ListFunc(ctx, opt.toListOptions())
		if err != nil {
			return err
		}

		return meta.EachListItem(list, fun)
	}
```
created_at: 2025-04-21T07:52:27Z
author_association: MEMBER
diff_hunk: @@ -189,6 +208,13 @@ func newDummyInformer[T runtime.Object](listWatch cache.ListerWatcher, opt Optio
 	return informer
 }
 
+func newDummyInformerWithWatchList[T runtime.Object](listWatch cache.ListerWatcher, opt Option, events chan<- Event[T]) *cache.Reflector {
path: pkg/utils/informer/informer.go
user: login: wzshiming
body: Remove this
created_at: 2025-04-21T09:24:00Z
author_association: MEMBER
diff_hunk: @@ -48,6 +49,24 @@ func (i *Informer[T, L]) Sync(ctx context.Context, opt Option, events chan<- Eve
 	if events == nil {
 		return fmt.Errorf("events channel is nil")
 	}
+
+	fun := func(obj runtime.Object) error {
path: pkg/utils/informer/informer.go
user: login: wzshiming
body: Reuse in `listPager.EachListItem(`
created_at: 2025-04-21T09:24:35Z
author_association: MEMBER
diff_hunk: @@ -153,6 +153,10 @@ type KwokConfigurationOptions struct {
 	// PodsOnNodeSyncParallelism is the number of workers to sync pods on nodes in parallel.
 	// +default=1
 	PodsOnNodeSyncParallelism uint `json:"podsOnNodeSyncParallelism,omitempty"`
+
+	// EnablePodsOnNodeSyncWatchList enables watch list for workers to sync pods on nodes.
+	// +default=false
+	EnablePodsOnNodeSyncWatchList *bool `json:"enablePodsOnNodeSyncWatchList"`
 }
path: pkg/apis/config/v1alpha1/kwok_configuration_types.go
user: login: wzshiming
body: I think would it be better to change it to EnablePodsOnNodeSyncNoListPager=false or EnablePodsOnNodeSyncListPager=true
created_at: 2025-04-21T09:27:49Z
author_association: MEMBER

PR comments:
user: login: netlify[bot]
created_at: 2025-04-21T03:33:33Z
body: ### <span aria-hidden="true">✅</span> Deploy Preview for *k8s-kwok* canceled.


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | f3e65d08a1cc3058c46625a4a5834a2f5312d961 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/k8s-kwok/deploys/6806322bcae6cd00084639a8 |
author_association: NONE
user: login: wzshiming
created_at: 2025-04-23T06:00:03Z
body: /approve
/lgtm

author_association: MEMBER
user: login: k8s-ci-robot
created_at: 2025-04-23T06:00:14Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/kwok/pull/1351#" title="Author self-approved">lianghao208</a>*, *<a href="https://github.com/kubernetes-sigs/kwok/pull/1351#issuecomment-2823150522" title="LGTM">wzshiming</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fkwok).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/kwok/blob/main/OWNERS)~~ [wzshiming]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: CONTRIBUTOR