Repository: kubernetes-sigs/kwok
Pull request #85 (2022-11-25 09:32:43)

refactor template cache
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?

/kind feature

#### What this PR does / why we need it:

It solves the problem of this parallel test. 

 #78

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
NONE
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/kwok/pull/85#" title="Author self-approved">carlory</a>*
**Once this PR has been reviewed and has the lgtm label**, please assign wzshiming for approval by writing `/assign @wzshiming` in a comment. For more information see [the Kubernetes Code Review Process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process).

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fkwok).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/kwok/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["wzshiming"]} -->
k8s-ci-robot: Hi @carlory. Thanks for your PR.

I'm waiting for a [kubernetes-sigs](https://github.com/orgs/kubernetes-sigs/people) member to verify that this patch is reasonable to test. If it is, they should reply with `/ok-to-test` on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should [join the org](https://git.k8s.io/community/community-membership.md#member) to skip this step.

Once the patch is verified, the new status will be reflected by the `ok-to-test` label.

I understand the commands that are listed [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fkwok).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>

wzshiming: /ok-to-test
carlory: cc @wzshiming 
k8s-ci-robot: @carlory: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kwok-unit-test-main | f8d3e2e361114af4844c890a77addbab2becff66 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/85/pull-kwok-unit-test-main/1596417708239360000) | true | `/test pull-kwok-unit-test-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kwok&pr=85). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Acarlory). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kwok/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @carlory: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kwok-unit-test-main | c4fa05adad0c2ab5c970dd9172af3b9cc66ace52 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kwok/85/pull-kwok-unit-test-main/1596418546575872000) | true | `/test pull-kwok-unit-test-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kwok&pr=85). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Acarlory). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kwok/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/.github/workflows/releases.yaml b/.github/workflows/releases.yaml
index ace41186..e887997f 100644
--- a/.github/workflows/releases.yaml
+++ b/.github/workflows/releases.yaml
@@ -17,7 +17,7 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v2
         with:
-          go-version: 1.19
+          go-version: 1.18
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
       - name: Set up Docker Buildx
diff --git a/go.mod b/go.mod
index 49185881..848463cf 100644
--- a/go.mod
+++ b/go.mod
@@ -1,6 +1,6 @@
 module sigs.k8s.io/kwok
 
-go 1.19
+go 1.18
 
 require (
 	github.com/nxadm/tail v1.4.4
diff --git a/pkg/kwok/controllers/node_controller.go b/pkg/kwok/controllers/node_controller.go
index 1cc3564f..033fe341 100644
--- a/pkg/kwok/controllers/node_controller.go
+++ b/pkg/kwok/controllers/node_controller.go
@@ -50,7 +50,7 @@ type NodeController struct {
 	nodesSets                             *stringSets
 	nodeHeartbeatTemplate                 string
 	nodeStatusTemplate                    string
-	funcMap                               template.FuncMap
+	renderer                              *renderer
 	logger                                logger.Logger
 	nodeHeartbeatInterval                 time.Duration
 	nodeHeartbeatParallelism              int
@@ -111,15 +111,15 @@ func NewNodeController(conf NodeControllerConfig) (*NodeController, error) {
 		lockNodeParallelism:                   conf.LockNodeParallelism,
 		nodeChan:                              make(chan string),
 	}
-	n.funcMap = template.FuncMap{
+	funcMap = template.FuncMap{
 		"NodeIP": func() string {
 			return n.nodeIP
 		},
 	}
 	for k, v := range conf.FuncMap {
-		n.funcMap[k] = v
+		funcMap[k] = v
 	}
-
+	n.renderer = newRenderer(funcMap)
 	return n, nil
 }
 
@@ -343,7 +343,7 @@ func (c *NodeController) LockNode(ctx context.Context, nodeName string) error {
 }
 
 func (c *NodeController) configureNode(node *corev1.Node) ([]byte, error) {
-	patch, err := toTemplateJson(c.nodeStatusTemplate, node, c.funcMap)
+	patch, err := c.renderer.renderToJson(c.nodeStatusTemplate, node)
 	if err != nil {
 		return nil, err
 	}
@@ -380,7 +380,7 @@ func (c *NodeController) configureNode(node *corev1.Node) ([]byte, error) {
 }
 
 func (c *NodeController) configureHeartbeatNode(node *corev1.Node) ([]byte, error) {
-	patch, err := toTemplateJson(c.nodeHeartbeatTemplate, node, c.funcMap)
+	patch, err := c.renderer.renderToJson(c.nodeHeartbeatTemplate, node)
 	if err != nil {
 		return nil, err
 	}
diff --git a/pkg/kwok/controllers/pod_controller.go b/pkg/kwok/controllers/pod_controller.go
index 81b8670c..e657d62b 100644
--- a/pkg/kwok/controllers/pod_controller.go
+++ b/pkg/kwok/controllers/pod_controller.go
@@ -57,7 +57,7 @@ type PodController struct {
 	ipPool                                *ipPool
 	podStatusTemplate                     string
 	logger                                logger.Logger
-	funcMap                               template.FuncMap
+	renderer                              *renderer
 	lockPodChan                           chan *corev1.Pod
 	lockPodParallelism                    int
 	deletePodChan                         chan *corev1.Pod
@@ -116,7 +116,7 @@ func NewPodController(conf PodControllerConfig) (*PodController, error) {
 		deletePodChan:                         make(chan *corev1.Pod),
 		deletePodParallelism:                  conf.DeletePodParallelism,
 	}
-	n.funcMap = template.FuncMap{
+	funcMap = template.FuncMap{
 		"NodeIP": func() string {
 			return n.nodeIP
 		},
@@ -125,8 +125,9 @@ func NewPodController(conf PodControllerConfig) (*PodController, error) {
 		},
 	}
 	for k, v := range conf.FuncMap {
-		n.funcMap[k] = v
+		funcMap[k] = v
 	}
+	n.renderer = newRenderer(funcMap)
 	return n, nil
 }
 
@@ -350,7 +351,7 @@ func (c *PodController) configurePod(pod *corev1.Pod) ([]byte, error) {
 		c.ipPool.Use(pod.Status.PodIP)
 	}
 
-	patch, err := configurePod(pod, c.podStatusTemplate, c.funcMap)
+	patch, err := c.computePatchData(pod, c.podStatusTemplate)
 	if err != nil {
 		return nil, err
 	}
@@ -363,8 +364,8 @@ func (c *PodController) configurePod(pod *corev1.Pod) ([]byte, error) {
 	})
 }
 
-func configurePod(pod *corev1.Pod, temp string, funcMap template.FuncMap) ([]byte, error) {
-	patch, err := toTemplateJson(temp, pod, funcMap)
+func (c *PodController) computePatchData(pod *corev1.Pod, temp string) ([]byte, error) {
+	patch, err := c.renderer.renderToJson(temp, pod)
 	if err != nil {
 		return nil, err
 	}
diff --git a/pkg/kwok/controllers/renderer.go b/pkg/kwok/controllers/renderer.go
new file mode 100644
index 00000000..8a428ac0
--- /dev/null
+++ b/pkg/kwok/controllers/renderer.go
@@ -0,0 +1,89 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package controllers
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"strings"
+	"sync"
+	"text/template"
+
+	"sigs.k8s.io/yaml"
+)
+
+type renderer struct {
+	cache      sync.Map
+	bufferPool sync.Pool
+	funcMap    template.FuncMap
+}
+
+func newRenderer(funcMap template.FuncMap) *renderer {
+	return &renderer{
+		funcMap: funcMap,
+		cache:   sync.Map{},
+		bufferPool: sync.Pool{
+			New: func() interface{} {
+				return bytes.NewBuffer(make([]byte, 4*1024))
+			},
+		},
+	}
+}
+
+// renderToJson renders the template with the given text and original object.
+func (r *renderer) renderToJson(text string, original interface{}) ([]byte, error) {
+	text = strings.TrimSpace(text)
+	v, ok := r.cache.Load(text)
+	if !ok {
+		temp, err := template.New("_").Funcs(r.funcMap).Parse(text)
+		if err != nil {
+			return nil, err
+		}
+		r.cache.Store(text, temp)
+		v = temp
+	}
+	temp := v.(*template.Template)
+	buf := r.bufferPool.Get().(*bytes.Buffer)
+	defer r.bufferPool.Put(buf)
+
+	buf.Reset()
+	err := json.NewEncoder(buf).Encode(original)
+	if err != nil {
+		return nil, err
+	}
+
+	var data interface{}
+	decoder := json.NewDecoder(buf)
+	decoder.UseNumber()
+	err = decoder.Decode(&data)
+	if err != nil {
+		return nil, err
+	}
+
+	buf.Reset()
+	err = temp.Execute(buf, data)
+	if err != nil {
+		return nil, err
+	}
+
+	out, err := yaml.YAMLToJSON(buf.Bytes())
+	if err != nil {
+		return nil, fmt.Errorf("%w: %s", err, buf.String())
+	}
+	return out, nil
+}
diff --git a/pkg/kwok/controllers/utils.go b/pkg/kwok/controllers/utils.go
index d09bd042..8fe53198 100644
--- a/pkg/kwok/controllers/utils.go
+++ b/pkg/kwok/controllers/utils.go
@@ -17,18 +17,12 @@ limitations under the License.
 package controllers
 
 import (
-	"bytes"
 	"encoding/binary"
-	"encoding/json"
-	"fmt"
 	"net"
-	"strings"
 	"sync"
-	"text/template"
 	"time"
 
 	"k8s.io/apimachinery/pkg/labels"
-	"sigs.k8s.io/yaml"
 )
 
 func parseCIDR(s string) (*net.IPNet, error) {
@@ -122,57 +116,6 @@ func (i *ipPool) Use(ip string) {
 	i.used[ip] = struct{}{}
 }
 
-func toTemplateJson(text string, original interface{}, funcMap template.FuncMap) ([]byte, error) {
-	text = strings.TrimSpace(text)
-	v, ok := templateCache.Load(text)
-	if !ok {
-		temp, err := template.New("_").Funcs(funcMap).Parse(text)
-		if err != nil {
-			return nil, err
-		}
-		templateCache.Store(text, temp)
-		v = temp
-	}
-	temp := v.(*template.Template)
-	buf := bufferPool.Get().(*bytes.Buffer)
-	defer bufferPool.Put(buf)
-
-	buf.Reset()
-	err := json.NewEncoder(buf).Encode(original)
-	if err != nil {
-		return nil, err
-	}
-
-	var data interface{}
-	decoder := json.NewDecoder(buf)
-	decoder.UseNumber()
-	err = decoder.Decode(&data)
-	if err != nil {
-		return nil, err
-	}
-
-	buf.Reset()
-	err = temp.Execute(buf, data)
-	if err != nil {
-		return nil, err
-	}
-
-	out, err := yaml.YAMLToJSON(buf.Bytes())
-	if err != nil {
-		return nil, fmt.Errorf("%w: %s", err, buf.String())
-	}
-	return out, nil
-}
-
-var (
-	templateCache = sync.Map{}
-	bufferPool    = sync.Pool{
-		New: func() interface{} {
-			return &bytes.Buffer{}
-		},
-	}
-)
-
 type parallelTasks struct {
 	wg     sync.WaitGroup
 	bucket chan struct{}


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
index 06ee4ef4..c1506d15 100644
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -16,7 +16,7 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v2
         with:
-          go-version: 1.19
+          go-version: 1.18
 
       - name: Install Kind
         shell: bash
@@ -69,7 +69,7 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v2
         with:
-          go-version: 1.19
+          go-version: 1.18
 
       - name: Install CoreUtils for MacOS
         if: ${{ matrix.os == 'macos-latest' && matrix.kwokctl-runtime != 'binary' }}
diff --git a/pkg/kwok/controllers/renderer_test.go b/pkg/kwok/controllers/renderer_test.go
new file mode 100644
index 00000000..ecb55cdd
--- /dev/null
+++ b/pkg/kwok/controllers/renderer_test.go
@@ -0,0 +1,82 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package controllers
+
+import (
+	"html/template"
+	"testing"
+)
+
+func TestRenderToJson(t *testing.T) {
+
+	testCases := []struct {
+		name      string
+		funcMap   template.FuncMap
+		templText string
+		original  interface{}
+		expected  string
+	}{
+		{
+			name:      "basic",
+			funcMap:   template.FuncMap{},
+			original:  map[string]interface{}{"k": "v1"},
+			templText: `{"k":{{ .k }}}`,
+			expected:  `{"k":"v1"}`,
+		},
+		{
+			name:      "basic with yaml format",
+			funcMap:   template.FuncMap{},
+			original:  map[string]interface{}{"k": "v1"},
+			templText: `k: {{ .k }}`,
+			expected:  `{"k":"v1"}`,
+		},
+		{
+			name: "with funcMap",
+			funcMap: template.FuncMap{
+				"Foo": func() string {
+					return "foo"
+				},
+			},
+			original:  map[string]interface{}{"k": "v1"},
+			templText: `{"foo":{{ Foo }},"k":{{ .k }}}`,
+			expected:  `{"foo":"foo","k":"v1"}`,
+		},
+		{
+			name: "with whitespace",
+			funcMap: template.FuncMap{
+				"Foo": func() string {
+					return "foo"
+				},
+			},
+			original:  map[string]interface{}{"k": "v1"},
+			templText: `        {"foo":{{ Foo }},"k":{{ .k }}}       `,
+			expected:  `{"foo":"foo","k":"v1"}`,
+		},
+	}
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			r := newRenderer(tc.funcMap)
+			actual, err := r.renderToJson(tc.templText, tc.original)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if string(actual) != tc.expected {
+				t.Fatalf("expected %s, got %s", tc.expected, actual)
+			}
+		})
+	}
+}