Repository: kubernetes-sigs/kwok
Pull request #749 (2023-08-01 10:58:25)

Tidy up this embedded CRs
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?

<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

/kind cleanup

#### What this PR does / why we need it:

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
NONE
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs
NONE
```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/kwok/pull/749#" title="Author self-approved">wzshiming</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fkwok).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/kwok/blob/main/OWNERS)~~ [wzshiming]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *k8s-kwok* processing.


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | e69fd4bf956a5487379539cb6c7baf601c4d1c4d |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/k8s-kwok/deploys/64c8e5535176cd00085e4d7f |
k8s-ci-robot: @wzshiming: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kwok-e2e-test-main | daf219c2ddf61e7e67ff4f9358372525eab9f648 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/749/pull-kwok-e2e-test-main/1686331030727823360) | true | `/test pull-kwok-e2e-test-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kwok&pr=749). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Awzshiming). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kwok/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @wzshiming: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kwok-e2e-test-main | daf219c2ddf61e7e67ff4f9358372525eab9f648 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/749/pull-kwok-e2e-test-main/1686331030727823360) | true | `/test pull-kwok-e2e-test-main`
pull-kwok-unit-test-main | 89db8dda329eeafbba9575a395f36e18746b3bf8 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/749/pull-kwok-unit-test-main/1686334158428704768) | true | `/test pull-kwok-unit-test-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kwok&pr=749). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Awzshiming). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kwok/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @wzshiming: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kwok-e2e-test-main | daf219c2ddf61e7e67ff4f9358372525eab9f648 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/749/pull-kwok-e2e-test-main/1686331030727823360) | true | `/test pull-kwok-e2e-test-main`
pull-kwok-unit-test-main | 89db8dda329eeafbba9575a395f36e18746b3bf8 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/749/pull-kwok-unit-test-main/1686334158428704768) | true | `/test pull-kwok-unit-test-main`
pull-kwok-verify-main | 89db8dda329eeafbba9575a395f36e18746b3bf8 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/749/pull-kwok-verify-main/1686334158340624384) | true | `/test pull-kwok-verify-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kwok&pr=749). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Awzshiming). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kwok/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/kustomize/stage/node/fast/embed.go b/kustomize/stage/node/fast/embed.go
new file mode 100644
index 00000000..c698ed76
--- /dev/null
+++ b/kustomize/stage/node/fast/embed.go
@@ -0,0 +1,28 @@
+/*
+Copyright 2023 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package fast contains the fast node for kwok.
+package fast
+
+import (
+	_ "embed"
+)
+
+var (
+	// DefaultNodeInit is the default node init yaml.
+	//go:embed node-initialize.yaml
+	DefaultNodeInit string
+)
diff --git a/kustomize/stage/node/heartbeat/embed.go b/kustomize/stage/node/heartbeat/embed.go
new file mode 100644
index 00000000..75f7a021
--- /dev/null
+++ b/kustomize/stage/node/heartbeat/embed.go
@@ -0,0 +1,28 @@
+/*
+Copyright 2023 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package heartbeat contains the node heartbeat for kwok.
+package heartbeat
+
+import (
+	_ "embed"
+)
+
+var (
+	// DefaultNodeHeartbeat is the default node heartbeat yaml.
+	//go:embed node-heartbeat.yaml
+	DefaultNodeHeartbeat string
+)
diff --git a/kustomize/stage/pod/fast/embed.go b/kustomize/stage/pod/fast/embed.go
new file mode 100644
index 00000000..859a36b1
--- /dev/null
+++ b/kustomize/stage/pod/fast/embed.go
@@ -0,0 +1,36 @@
+/*
+Copyright 2023 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package fast contains the fast pod for kwok.
+package fast
+
+import (
+	_ "embed"
+)
+
+var (
+	// DefaultPodReady is the default pod ready yaml.
+	//go:embed pod-ready.yaml
+	DefaultPodReady string
+
+	// DefaultPodComplete is the default pod complete yaml.
+	//go:embed pod-complete.yaml
+	DefaultPodComplete string
+
+	// DefaultPodDelete is the default pod delete yaml.
+	//go:embed pod-delete.yaml
+	DefaultPodDelete string
+)
diff --git a/pkg/config/config.go b/pkg/config/config.go
index 2e0180ac..7f50023b 100644
--- a/pkg/config/config.go
+++ b/pkg/config/config.go
@@ -42,6 +42,10 @@ import (
 	"sigs.k8s.io/kwok/pkg/utils/path"
 )
 
+var (
+	errUnsupportedType = errors.New("unsupported type")
+)
+
 func loadRawMessages(src []string) ([]json.RawMessage, error) {
 	var raws []json.RawMessage
 
@@ -366,34 +370,14 @@ func SaveTo(ctx context.Context, w io.Writer, objs []InternalObject) error {
 			}
 		}
 
-		typ := reflect.TypeOf(obj)
-		if typ.Kind() == reflect.Ptr {
-			typ = typ.Elem()
-		}
-		kind := typ.Name()
-		handler, ok := configHandlers[kind]
-		if !ok {
-			logger.Warn("Unsupported type",
-				"type", kind,
-			)
-			continue
-		}
-
-		versiondObj, err := handler.MutateToVersiond([]InternalObject{obj})
-		if err != nil {
-			return err
-		}
-		if len(versiondObj) != 1 {
-			return fmt.Errorf("unexpected length of versiond object: %d", len(versiondObj))
-		}
-
-		data, err := handler.Marshal(versiondObj[0])
-		if err != nil {
-			return err
-		}
-
-		data, err = yaml.JSONToYAML(data)
+		data, err := Marshal(obj)
 		if err != nil {
+			if errors.Is(err, errUnsupportedType) {
+				logger.Warn("Unsupported type", err,
+					"obj", obj,
+				)
+				continue
+			}
 			return err
 		}
 
@@ -405,6 +389,80 @@ func SaveTo(ctx context.Context, w io.Writer, objs []InternalObject) error {
 	return nil
 }
 
+// Unmarshal unmarshals the given raw message into the internal object.
+func Unmarshal(raw []byte) (InternalObject, error) {
+	meta := metav1.TypeMeta{}
+
+	raw, err := yaml.YAMLToJSON(raw)
+	if err != nil {
+		return nil, err
+	}
+
+	err = json.Unmarshal(raw, &meta)
+	if err != nil {
+		return nil, err
+	}
+
+	gvk := meta.GroupVersionKind()
+
+	handler, ok := configHandlers[gvk.Kind]
+	if !ok {
+		return nil, fmt.Errorf("%w: %s", errUnsupportedType, gvk.Kind)
+	}
+
+	vobj, err := handler.Unmarshal(raw)
+	if err != nil {
+		return nil, fmt.Errorf("failed to unmarshal %s: %w", gvk.Kind, err)
+	}
+
+	iobj, err := handler.MutateToInternal([]versiondObject{vobj})
+	if err != nil {
+		return nil, fmt.Errorf("failed to convert %s: %w", gvk.Kind, err)
+	}
+
+	if len(iobj) == 0 {
+		return nil, fmt.Errorf("failed to convert %s: no object", gvk.Kind)
+	}
+
+	if len(iobj) > 1 {
+		return nil, fmt.Errorf("failed to convert %s: too many objects", gvk.Kind)
+	}
+
+	return iobj[0], nil
+}
+
+// Marshal marshals the given internal object into a raw message.
+func Marshal(obj InternalObject) ([]byte, error) {
+	typ := reflect.TypeOf(obj)
+	if typ.Kind() == reflect.Ptr {
+		typ = typ.Elem()
+	}
+	kind := typ.Name()
+	handler, ok := configHandlers[kind]
+	if !ok {
+		return nil, fmt.Errorf("%w: %s", errUnsupportedType, kind)
+	}
+
+	versiondObj, err := handler.MutateToVersiond([]InternalObject{obj})
+	if err != nil {
+		return nil, err
+	}
+	if len(versiondObj) != 1 {
+		return nil, fmt.Errorf("unexpected length of versiond object: %d", len(versiondObj))
+	}
+
+	data, err := handler.Marshal(versiondObj[0])
+	if err != nil {
+		return nil, err
+	}
+
+	data, err = yaml.JSONToYAML(data)
+	if err != nil {
+		return nil, err
+	}
+	return data, nil
+}
+
 // FilterWithType returns a list of objects with the given type.
 func FilterWithType[T InternalObject](objs []InternalObject) (out []T) {
 	for _, obj := range objs {
diff --git a/pkg/kwok/cmd/root.go b/pkg/kwok/cmd/root.go
index 7f48a506..4b3aec50 100644
--- a/pkg/kwok/cmd/root.go
+++ b/pkg/kwok/cmd/root.go
@@ -28,6 +28,9 @@ import (
 	"k8s.io/client-go/kubernetes"
 	"k8s.io/utils/clock"
 
+	nodefast "sigs.k8s.io/kwok/kustomize/stage/node/fast"
+	nodeheartbeat "sigs.k8s.io/kwok/kustomize/stage/node/heartbeat"
+	podfast "sigs.k8s.io/kwok/kustomize/stage/pod/fast"
 	"sigs.k8s.io/kwok/pkg/apis/internalversion"
 	"sigs.k8s.io/kwok/pkg/apis/v1alpha1"
 	"sigs.k8s.io/kwok/pkg/config"
@@ -41,7 +44,6 @@ import (
 	"sigs.k8s.io/kwok/pkg/utils/slices"
 	"sigs.k8s.io/kwok/pkg/utils/version"
 	"sigs.k8s.io/kwok/pkg/utils/wait"
-	"sigs.k8s.io/kwok/stages"
 )
 
 type flagpole struct {
@@ -140,22 +142,14 @@ func runE(ctx context.Context, flags *flagpole) error {
 	if !slices.Contains(flags.Options.EnableCRDs, v1alpha1.StageKind) {
 		if len(nodeStages) == 0 {
 			logger.Warn("No node stages found, using default node stages")
-			nodeStages, err = controllers.NewStagesFromYaml([]byte(stages.DefaultNodeStages))
+			nodeStages, err = getDefaultNodeStages(flags.Options.NodeLeaseDurationSeconds == 0)
 			if err != nil {
 				return err
 			}
-			if flags.Options.NodeLeaseDurationSeconds == 0 {
-				nodeHeartbeatStages, err := controllers.NewStagesFromYaml([]byte(stages.DefaultNodeHeartbeatStages))
-				if err != nil {
-					return err
-				}
-				nodeStages = append(nodeStages, nodeHeartbeatStages...)
-			}
 		}
 
 		if len(podStages) == 0 {
-			logger.Warn("No pod stages found, using default pod stages")
-			podStages, err = controllers.NewStagesFromYaml([]byte(stages.DefaultPodStages))
+			podStages, err = getDefaultPodStages()
 			if err != nil {
 				return err
 			}
@@ -390,3 +384,48 @@ func waitForReady(ctx context.Context, clientset kubernetes.Interface) error {
 	}
 	return nil
 }
+
+func getDefaultNodeStages(heartbeat bool) ([]*internalversion.Stage, error) {
+	nodeStages := []*internalversion.Stage{}
+	nodeInit, err := config.Unmarshal([]byte(nodefast.DefaultNodeInit))
+	if err != nil {
+		return nil, err
+	}
+	nodeInitStage, ok := nodeInit.(*internalversion.Stage)
+	if !ok {
+		return nil, fmt.Errorf("failed to convert node init to stage")
+	}
+	nodeStages = append(nodeStages, nodeInitStage)
+
+	if heartbeat {
+		nodeHeartbeat, err := config.Unmarshal([]byte(nodeheartbeat.DefaultNodeHeartbeat))
+		if err != nil {
+			return nil, err
+		}
+		nodeHeartbeatStage, ok := nodeHeartbeat.(*internalversion.Stage)
+		if !ok {
+			return nil, fmt.Errorf("failed to convert node init to stage")
+		}
+		nodeStages = append(nodeStages, nodeHeartbeatStage)
+	}
+	return nodeStages, nil
+}
+
+func getDefaultPodStages() ([]*internalversion.Stage, error) {
+	return slices.MapWithError([]string{
+		podfast.DefaultPodReady,
+		podfast.DefaultPodComplete,
+		podfast.DefaultPodDelete,
+	}, func(s string) (*internalversion.Stage, error) {
+		iobj, err := config.Unmarshal([]byte(s))
+		if err != nil {
+			return nil, err
+		}
+		stage, ok := iobj.(*internalversion.Stage)
+		if !ok {
+			return nil, fmt.Errorf("failed to convert pod init to stage")
+		}
+
+		return stage, nil
+	})
+}
diff --git a/pkg/kwok/controllers/lifecycle.go b/pkg/kwok/controllers/lifecycle.go
index 3780fb16..937b0b73 100644
--- a/pkg/kwok/controllers/lifecycle.go
+++ b/pkg/kwok/controllers/lifecycle.go
@@ -17,46 +17,18 @@ limitations under the License.
 package controllers
 
 import (
-	"bytes"
 	"context"
-	"errors"
 	"fmt"
-	"io"
 	"math/rand"
 	"time"
 
 	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apimachinery/pkg/util/yaml"
 
 	"sigs.k8s.io/kwok/pkg/apis/internalversion"
-	"sigs.k8s.io/kwok/pkg/apis/v1alpha1"
 	"sigs.k8s.io/kwok/pkg/utils/expression"
 	"sigs.k8s.io/kwok/pkg/utils/format"
 )
 
-// NewStagesFromYaml returns stages from yaml data.
-func NewStagesFromYaml(data []byte) ([]*internalversion.Stage, error) {
-	var stages []*internalversion.Stage
-	decoder := yaml.NewYAMLToJSONDecoder(bytes.NewBuffer(data))
-	for {
-		var stage v1alpha1.Stage
-		err := decoder.Decode(&stage)
-		if err != nil {
-			if errors.Is(err, io.EOF) {
-				break
-			}
-			return nil, err
-		}
-
-		internalStage, err := internalversion.ConvertToInternalStage(&stage)
-		if err != nil {
-			return nil, err
-		}
-		stages = append(stages, internalStage)
-	}
-	return stages, nil
-}
-
 // NewLifecycle returns a new Lifecycle.
 func NewLifecycle(stages []*internalversion.Stage) (Lifecycle, error) {
 	lcs := Lifecycle{}
diff --git a/pkg/kwokctl/cmd/scale/scale.go b/pkg/kwokctl/cmd/scale/scale.go
index 082f50c5..61f85c6a 100644
--- a/pkg/kwokctl/cmd/scale/scale.go
+++ b/pkg/kwokctl/cmd/scale/scale.go
@@ -25,7 +25,6 @@ import (
 	"path"
 
 	"github.com/spf13/cobra"
-	utilyaml "k8s.io/apimachinery/pkg/util/yaml"
 
 	"sigs.k8s.io/kwok/kustomize/kwokctl/resource"
 	"sigs.k8s.io/kwok/pkg/apis/internalversion"
@@ -99,19 +98,24 @@ func runE(ctx context.Context, flags *flagpole, args []string) error {
 		return krc.Name == resourceKind
 	})
 	if !ok {
+		var resourceData []byte
 		switch resourceKind {
 		default:
 			return fmt.Errorf("resource %s is not exists", resourceKind)
 		case "pod":
-			err = utilyaml.Unmarshal([]byte(resource.DefaultPod), &krc)
-			if err != nil {
-				return err
-			}
+			resourceData = []byte(resource.DefaultPod)
 		case "node":
-			err = utilyaml.Unmarshal([]byte(resource.DefaultNode), &krc)
-			if err != nil {
-				return err
-			}
+			resourceData = []byte(resource.DefaultNode)
+		}
+
+		logger.Info("No resource found, use default resource", "resource", resourceKind)
+		iobj, err := config.Unmarshal(resourceData)
+		if err != nil {
+			return err
+		}
+		krc, ok = iobj.(*internalversion.KwokctlResource)
+		if !ok {
+			return fmt.Errorf("resource %T is not a kwokctl resource", iobj)
 		}
 	}
 
diff --git a/pkg/kwokctl/runtime/cluster.go b/pkg/kwokctl/runtime/cluster.go
index 6f5dde45..eef3161d 100644
--- a/pkg/kwokctl/runtime/cluster.go
+++ b/pkg/kwokctl/runtime/cluster.go
@@ -27,11 +27,12 @@ import (
 	"github.com/nxadm/tail"
 
 	"sigs.k8s.io/kwok/kustomize/crd"
+	nodefast "sigs.k8s.io/kwok/kustomize/stage/node/fast"
+	heartbeat "sigs.k8s.io/kwok/kustomize/stage/node/heartbeat"
 	"sigs.k8s.io/kwok/pkg/apis/internalversion"
 	"sigs.k8s.io/kwok/pkg/apis/v1alpha1"
 	"sigs.k8s.io/kwok/pkg/config"
 	"sigs.k8s.io/kwok/pkg/consts"
-	"sigs.k8s.io/kwok/pkg/kwok/controllers"
 	"sigs.k8s.io/kwok/pkg/kwokctl/dryrun"
 	"sigs.k8s.io/kwok/pkg/kwokctl/snapshot"
 	"sigs.k8s.io/kwok/pkg/log"
@@ -42,7 +43,6 @@ import (
 	"sigs.k8s.io/kwok/pkg/utils/slices"
 	"sigs.k8s.io/kwok/pkg/utils/version"
 	"sigs.k8s.io/kwok/pkg/utils/wait"
-	"sigs.k8s.io/kwok/stages"
 )
 
 // The following functions are used to get the path of the cluster
@@ -201,34 +201,32 @@ func (c *Cluster) Save(ctx context.Context) error {
 
 func (c *Cluster) getDefaultStages(updateFrequency int64, nodeHeartbeat bool) ([]config.InternalObject, error) {
 	objs := []config.InternalObject{}
-	nodeStages, err := controllers.NewStagesFromYaml([]byte(stages.DefaultNodeStages))
+
+	nodeInit, err := config.Unmarshal([]byte(nodefast.DefaultNodeInit))
 	if err != nil {
 		return nil, err
 	}
-	for _, stage := range nodeStages {
-		objs = append(objs, stage)
+	if _, ok := nodeInit.(*internalversion.Stage); !ok {
+		return nil, fmt.Errorf("failed to get node init stage %T", nodeInit)
 	}
+	objs = append(objs, nodeInit)
 
 	if nodeHeartbeat {
-		nodeHeartbeatStages, err := controllers.NewStagesFromYaml([]byte(stages.DefaultNodeHeartbeatStages))
+		nodeHeartbeat, err := config.Unmarshal([]byte(heartbeat.DefaultNodeHeartbeat))
 		if err != nil {
 			return nil, err
 		}
 
+		nodeHeartbeatStage, ok := nodeHeartbeat.(*internalversion.Stage)
+		if !ok {
+			return nil, fmt.Errorf("failed to get node heartbeat stage %T", nodeHeartbeat)
+		}
 		if updateFrequency > 0 {
-			hasUpdate := false
-			for _, stage := range nodeHeartbeatStages {
-				if stage.Name == "node-heartbeat" {
-					stage.Spec.Delay.DurationMilliseconds = format.Ptr(updateFrequency)
-					stage.Spec.Delay.JitterDurationMilliseconds = format.Ptr(updateFrequency + updateFrequency/10)
-					hasUpdate = true
-				}
-				objs = append(objs, stage)
-			}
-			if !hasUpdate {
-				return nil, fmt.Errorf("failed to update node heartbeat stage")
-			}
+			nodeHeartbeatStage.Spec.Delay.DurationMilliseconds = format.Ptr(updateFrequency)
+			nodeHeartbeatStage.Spec.Delay.JitterDurationMilliseconds = format.Ptr(updateFrequency + updateFrequency/10)
 		}
+
+		objs = append(objs, nodeHeartbeatStage)
 	}
 	return objs, nil
 }
diff --git a/pkg/utils/slices/slices.go b/pkg/utils/slices/slices.go
index 71a24b95..d69f3650 100644
--- a/pkg/utils/slices/slices.go
+++ b/pkg/utils/slices/slices.go
@@ -25,6 +25,20 @@ func Map[S ~[]T, T any, O any](s S, f func(T) O) []O {
 	return out
 }
 
+// MapWithError returns a new slice containing the results of applying the given function
+// to all elements in the slice that satisfy the predicate f.
+func MapWithError[S ~[]T, T any, O any](s S, f func(T) (O, error)) ([]O, error) {
+	out := make([]O, len(s))
+	for i := range s {
+		o, err := f(s[i])
+		if err != nil {
+			return nil, err
+		}
+		out[i] = o
+	}
+	return out, nil
+}
+
 // Find returns the first element in the slice that satisfies the predicate f.
 func Find[S ~[]T, T any](s S, f func(T) bool) (t T, ok bool) {
 	for _, v := range s {
diff --git a/stages/README.md b/stages/README.md
new file mode 100644
index 00000000..c98d238c
--- /dev/null
+++ b/stages/README.md
@@ -0,0 +1,3 @@
+# Stages (DEPRECATED)
+
+Use ./kustomize/stage directory instead of this directory. this directory will be removed in the future.


Test patch:
diff --git a/pkg/kwok/controllers/controller_test.go b/pkg/kwok/controllers/controller_test.go
index 56774593..441dc312 100644
--- a/pkg/kwok/controllers/controller_test.go
+++ b/pkg/kwok/controllers/controller_test.go
@@ -28,9 +28,13 @@ import (
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/client-go/kubernetes/fake"
 
+	nodefast "sigs.k8s.io/kwok/kustomize/stage/node/fast"
+	podfast "sigs.k8s.io/kwok/kustomize/stage/pod/fast"
+	"sigs.k8s.io/kwok/pkg/apis/internalversion"
+	"sigs.k8s.io/kwok/pkg/config"
 	"sigs.k8s.io/kwok/pkg/log"
+	"sigs.k8s.io/kwok/pkg/utils/slices"
 	"sigs.k8s.io/kwok/pkg/utils/wait"
-	"sigs.k8s.io/kwok/stages"
 )
 
 func TestController(t *testing.T) {
@@ -67,10 +71,19 @@ func TestController(t *testing.T) {
 		},
 	}
 
-	nodeStages, _ := NewStagesFromYaml([]byte(stages.DefaultNodeStages))
-	nodeHeartbeatStages, _ := NewStagesFromYaml([]byte(stages.DefaultNodeHeartbeatStages))
-	nodeStages = append(nodeStages, nodeHeartbeatStages...)
-	podStages, _ := NewStagesFromYaml([]byte(stages.DefaultPodStages))
+	nodeInit, _ := config.Unmarshal([]byte(nodefast.DefaultNodeInit))
+	nodeStages := []*internalversion.Stage{nodeInit.(*internalversion.Stage)}
+	podStages, _ := slices.MapWithError([]string{
+		podfast.DefaultPodReady,
+		podfast.DefaultPodComplete,
+		podfast.DefaultPodDelete,
+	}, func(s string) (*internalversion.Stage, error) {
+		iobj, err := config.Unmarshal([]byte(s))
+		if err != nil {
+			return nil, err
+		}
+		return iobj.(*internalversion.Stage), nil
+	})
 
 	tests := []struct {
 		name          string
diff --git a/pkg/kwok/controllers/lifecycle_test.go b/pkg/kwok/controllers/lifecycle_test.go
deleted file mode 100644
index e2a07c48..00000000
--- a/pkg/kwok/controllers/lifecycle_test.go
+++ /dev/null
@@ -1,78 +0,0 @@
-/*
-Copyright 2023 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package controllers
-
-import (
-	"testing"
-
-	"github.com/google/go-cmp/cmp"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-
-	"sigs.k8s.io/kwok/pkg/apis/internalversion"
-)
-
-func TestNewStagesFromYaml(t *testing.T) {
-	type args struct {
-		data []byte
-	}
-	tests := []struct {
-		name    string
-		args    args
-		want    []*internalversion.Stage
-		wantErr bool
-	}{
-		{
-			name: "empty",
-			args: args{
-				data: []byte(`kind: Stage
-apiVersion: kwok.x-k8s.io/v1alpha1
-metadata:
-  name: node-test
-spec:
-  resourceRef:
-    apiGroup: v1
-    kind: Node`),
-			},
-			want: []*internalversion.Stage{
-				{
-					ObjectMeta: metav1.ObjectMeta{
-						Name: "node-test",
-					},
-					Spec: internalversion.StageSpec{
-						ResourceRef: internalversion.StageResourceRef{
-							APIGroup: "v1",
-							Kind:     "Node",
-						},
-					},
-				},
-			},
-		},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			got, err := NewStagesFromYaml(tt.args.data)
-			if (err != nil) != tt.wantErr {
-				t.Errorf("NewStagesFromYaml() error = %v, wantErr %v", err, tt.wantErr)
-				return
-			}
-
-			if diff := cmp.Diff(got, tt.want); diff != "" {
-				t.Errorf("NewStagesFromYaml() diff (-got +want): %s", diff)
-			}
-		})
-	}
-}
diff --git a/pkg/kwok/controllers/node_controller_test.go b/pkg/kwok/controllers/node_controller_test.go
index d37dbbec..8b86f785 100644
--- a/pkg/kwok/controllers/node_controller_test.go
+++ b/pkg/kwok/controllers/node_controller_test.go
@@ -29,10 +29,12 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/kubernetes/fake"
 
+	nodefast "sigs.k8s.io/kwok/kustomize/stage/node/fast"
+	"sigs.k8s.io/kwok/pkg/apis/internalversion"
+	"sigs.k8s.io/kwok/pkg/config"
 	"sigs.k8s.io/kwok/pkg/config/resources"
 	"sigs.k8s.io/kwok/pkg/log"
 	"sigs.k8s.io/kwok/pkg/utils/wait"
-	"sigs.k8s.io/kwok/stages"
 )
 
 func TestNodeController(t *testing.T) {
@@ -69,9 +71,10 @@ func TestNodeController(t *testing.T) {
 	nodeSelectorFunc := func(node *corev1.Node) bool {
 		return strings.HasPrefix(node.Name, "node")
 	}
-	nodeStages, _ := NewStagesFromYaml([]byte(stages.DefaultNodeStages))
-	nodeHeartbeatStages, _ := NewStagesFromYaml([]byte(stages.DefaultNodeHeartbeatStages))
-	nodeStages = append(nodeStages, nodeHeartbeatStages...)
+
+	nodeInit, _ := config.Unmarshal([]byte(nodefast.DefaultNodeInit))
+	nodeStages := []*internalversion.Stage{nodeInit.(*internalversion.Stage)}
+
 	lifecycle, _ := NewLifecycle(nodeStages)
 	nodes, err := NewNodeController(NodeControllerConfig{
 		TypedClient:          clientset,
diff --git a/pkg/kwok/controllers/pod_controller_test.go b/pkg/kwok/controllers/pod_controller_test.go
index 511c740a..c0ccfcea 100644
--- a/pkg/kwok/controllers/pod_controller_test.go
+++ b/pkg/kwok/controllers/pod_controller_test.go
@@ -29,11 +29,13 @@ import (
 	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/client-go/kubernetes/fake"
 
+	podfast "sigs.k8s.io/kwok/kustomize/stage/pod/fast"
+	"sigs.k8s.io/kwok/pkg/apis/internalversion"
+	"sigs.k8s.io/kwok/pkg/config"
 	"sigs.k8s.io/kwok/pkg/config/resources"
 	"sigs.k8s.io/kwok/pkg/log"
 	"sigs.k8s.io/kwok/pkg/utils/slices"
 	"sigs.k8s.io/kwok/pkg/utils/wait"
-	"sigs.k8s.io/kwok/stages"
 )
 
 const (
@@ -163,7 +165,18 @@ func TestPodController(t *testing.T) {
 		return nodeInfo, true
 	}
 
-	podStages, _ := NewStagesFromYaml([]byte(stages.DefaultPodStages))
+	podStages, _ := slices.MapWithError([]string{
+		podfast.DefaultPodReady,
+		podfast.DefaultPodComplete,
+		podfast.DefaultPodDelete,
+	}, func(s string) (*internalversion.Stage, error) {
+		iobj, err := config.Unmarshal([]byte(s))
+		if err != nil {
+			return nil, err
+		}
+		return iobj.(*internalversion.Stage), nil
+	})
+
 	lifecycle, _ := NewLifecycle(podStages)
 	annotationSelector, _ := labels.Parse("fake=custom")
 	pods, err := NewPodController(PodControllerConfig{