Repository: containerd/containerd
Pull request #11311 (2025-01-27 02:39:32)

Support multiple cni plugin bin dirs
Fix https://github.com/containerd/containerd/issues/10710, this PR adds a new cri CNI config `bin_dirs` that is similar to `bin_dir` but accepts a list of bin dirs. This PR includes:

commit 1: support his new `bin_dirs` field:
1. Add this new `bin_dirs` field;
2. In `ValidateRuntimeConfig`, make sure only one of `bin_dir` and `bin_dirs` is set.

commit 2: make `bin_dirs` as default, and migrate `bin_dir` to `bin_dirs`:
1. Change default to setting `bin_dirs` instead of `bin_dir`.
2. Mark `bin_dir` as DEPRECATED.
3. In ConfigMigration, migrate `bin_dir` in version 2 to `bin_dirs` in version 3.
4. Add a deprecation warning for `bin_dir`.




k8s-ci-robot: Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with `/test all`
k8s-ci-robot: @djdongjin: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-containerd-node-e2e | d248d2df5c47dcca8d62b60c11606ed8b743c99b | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/containerd_containerd/11311/pull-containerd-node-e2e/1887186280370933760) | true | `/test pull-containerd-node-e2e`

[Full PR test history](https://prow.k8s.io/pr-history?org=containerd&repo=containerd&pr=11311). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adjdongjin). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/containerd/containerd/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
djdongjin: > Let's add the validation and do some integration testing

@mikebrow Thanks for the review and suggestion, will add tests to verify the config-side change and the new behavior of supporting multiple bin dirs.

Also will mark `NetworkPluginBinDir` as `Deprecated` and migrate its usage in containerd :)
MikeZappa87: > trying to find the reason we never did this..
> 
> anyhow I don't see a good reason to not support it outside we've been talking about switching to rpc and this would be an intermediate step in that case.
> 
> Let's add the validation and do some integration testing to make sure it's all tied in well with the go-cni and cni libs should work as cni already has the plugin dir list but still let's test it

I remember us talking about this years ago. I’ll take a look at this once I’m home 
MikeZappa87: This looks good. Not sure why we didn't do this earlier? Since this function already takes a slice of string. https://github.com/containerd/go-cni/blob/f4736bb1d1b10293d9c484cbf79254a5efa1e020/opts.go#L44
k8s-ci-robot: @djdongjin: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-containerd-node-e2e | 089224bb65670642fc6ff28360ce9b44eb346c5a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/containerd_containerd/11311/pull-containerd-node-e2e/1888126313651245056) | true | `/test pull-containerd-node-e2e`

[Full PR test history](https://prow.k8s.io/pr-history?org=containerd&repo=containerd&pr=11311). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adjdongjin). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/containerd/containerd/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
djdongjin: /retest
k8s-ci-robot: @djdongjin: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-containerd-node-e2e | 089224bb65670642fc6ff28360ce9b44eb346c5a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/containerd_containerd/11311/pull-containerd-node-e2e/1888997997207883776) | true | `/test pull-containerd-node-e2e`

[Full PR test history](https://prow.k8s.io/pr-history?org=containerd&repo=containerd&pr=11311). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adjdongjin). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/containerd/containerd/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
djdongjin: /retest
k8s-ci-robot: @djdongjin: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-containerd-node-e2e | 089224bb65670642fc6ff28360ce9b44eb346c5a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/containerd_containerd/11311/pull-containerd-node-e2e/1891632248457793536) | true | `/test pull-containerd-node-e2e`

[Full PR test history](https://prow.k8s.io/pr-history?org=containerd&repo=containerd&pr=11311). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adjdongjin). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/containerd/containerd/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
djdongjin: /retest
k8s-ci-robot: @djdongjin: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-containerd-k8s-e2e-ec2 | 3f3fca2d9a0c19497de0d9df8eecf93101d33772 | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/containerd_containerd/11311/pull-containerd-k8s-e2e-ec2/1897840292501917696) | false | `/test pull-containerd-k8s-e2e-ec2`

[Full PR test history](https://prow.k8s.io/pr-history?org=containerd&repo=containerd&pr=11311). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adjdongjin). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/containerd/containerd/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
djdongjin: > and I think we should update the defaults that we use if there is no config.toml and or that we output on containerd config default:

@mikebrow thanks for the comments and code pointers. I added a 2nd commit to resolve the comments. It includes (1). update the default config value; (2). add deprecated note in `config.go`; (3). config migration from version 2 to version 3.

(3) is needed because now version 3 uses `bin_dirs` as the default whereas version 2 still uses `bin_dir`.

PTAL at your convenience. thanks!
djdongjin: > another spot.. 

https://github.com/containerd/containerd/blob/937e32836ffe0e63df43db497c21e2890dc062a2/docs/cri/config.md?plain=1#L578-L579

I updated the other 3 places but kept this one unchanged, as this one is for version 2 config and now we're updating version 3 config.
mikebrow: @samuelkarp pls check over the migration
cc @MikeZappa87 @fuweid 
djdongjin: > @samuelkarp pls check over the migration

Hi @samuelkarp any other comments/suggestions on this PR? thanks!

Patch:
diff --git a/RELEASES.md b/RELEASES.md
index 96158b956..1a0e517ee 100644
--- a/RELEASES.md
+++ b/RELEASES.md
@@ -469,6 +469,7 @@ The deprecated features are shown in the following table:
 | CRI `v1alpha2`                                                                   | containerd v1.7     | containerd v2.0 ✅                    | Use CRI `v1`                             |
 | Legacy CRI implementation of podsandbox support                                  | containerd v2.0     | containerd v2.0 ✅                    |                                          |
 | Go-Plugin library (`*.so`) as containerd runtime plugin                          | containerd v2.0     | containerd v2.1                       | Use external plugins (proxy or binary)   |
+| CNI `bin_dir` in CRI runtime config (`plugins.'io.containerd.cri.v1.runtime'.cni.bin_dir`) | containerd v2.1     | containerd v2.2                       | Change `bin_dir` to `bin_dirs` in the same section which supports a list of directories  |
 
 - Pulling Schema 1 images has been disabled in containerd v2.0, but it still can be enabled by setting an environment variable `CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE=1`
   until containerd v2.1. `ctr` users have to specify `--local` too (e.g., `ctr images pull --local`). Users of CRI clients (such as Kubernetes and `crictl`) have to specify this environment variable on the containerd daemon (usually in the systemd unit).
diff --git a/docs/cri/config.md b/docs/cri/config.md
index 77c93b555..bc03936c0 100644
--- a/docs/cri/config.md
+++ b/docs/cri/config.md
@@ -247,7 +247,9 @@ version = 3
             ShimCgroup = ''
 
     [plugins.'io.containerd.cri.v1.runtime'.cni]
-      bin_dir = '/opt/cni/bin'
+      # DEPRECATED, use `bin_dirs` instead (since containerd v2.1).
+      bin_dir = ''
+      bin_dirs = ['/opt/cni/bin']
       conf_dir = '/etc/cni/net.d'
       max_conf_num = 1
       setup_serially = false
diff --git a/internal/cri/config/config.go b/internal/cri/config/config.go
index 2a78f1660..83a49055c 100644
--- a/internal/cri/config/config.go
+++ b/internal/cri/config/config.go
@@ -21,6 +21,7 @@ import (
 	"errors"
 	"fmt"
 	"net/url"
+	"slices"
 	"time"
 
 	"github.com/containerd/log"
@@ -152,7 +153,13 @@ type ContainerdConfig struct {
 // CniConfig contains toml config related to cni
 type CniConfig struct {
 	// NetworkPluginBinDir is the directory in which the binaries for the plugin is kept.
+	//
+	// DEPRECATED: use `NetworkPluginBinDirs` instead.`
 	NetworkPluginBinDir string `toml:"bin_dir" json:"binDir"`
+	// NetworkPluginBinDirs is the directories in which the binaries for the plugin is kept.
+	//
+	// Only use one of NetworkPluginBinDir and NetworkPluginBinDirs, not both.
+	NetworkPluginBinDirs []string `toml:"bin_dirs" json:"binDirs"`
 	// NetworkPluginConfDir is the directory in which the admin places a CNI conf.
 	NetworkPluginConfDir string `toml:"conf_dir" json:"confDir"`
 	// NetworkPluginMaxConfNum is the max number of plugin config files that will
@@ -528,6 +535,28 @@ func ValidateRuntimeConfig(ctx context.Context, c *RuntimeConfig) ([]deprecation
 		return warnings, fmt.Errorf("no corresponding runtime configured in `containerd.runtimes` for `containerd` `default_runtime_name = \"%s\"", c.ContainerdConfig.DefaultRuntimeName)
 	}
 
+	// Validation for CNI config
+	if len(c.CniConfig.NetworkPluginBinDir) != 0 {
+		warnings = append(warnings, deprecation.CRICNIBinDir)
+		log.G(ctx).Warning("`bin_dir` is deprecated, please use `bin_dirs` instead")
+
+		if slices.Equal(c.CniConfig.NetworkPluginBinDirs, defaultNetworkPluginBinDirs()) {
+			// if a user set `bin_dir` explicitly, we remove the default value of `bin_dirs`
+			// to avoid the unexpected conflict between the two since we don't allow setting both.
+			c.CniConfig.NetworkPluginBinDirs = nil
+		}
+		if len(c.CniConfig.NetworkPluginBinDirs) == 0 {
+			// Before `NetworkPluginBinDir` is deprecated and removed, we manually move it
+			// into `NetworkPluginBinDirs` (if `NetworkPluginBinDirs` is empty)
+			// so that we can use it in the rest of the code.
+			c.CniConfig.NetworkPluginBinDirs = []string{c.CniConfig.NetworkPluginBinDir}
+			c.CniConfig.NetworkPluginBinDir = ""
+		}
+	}
+	if len(c.CniConfig.NetworkPluginBinDirs) != 0 && len(c.CniConfig.NetworkPluginBinDir) != 0 {
+		return warnings, errors.New("`cni.bin_dir` and `cni.bin_dirs` cannot be set at the same time")
+	}
+
 	for k, r := range c.ContainerdConfig.Runtimes {
 		if r.CgroupWritable && !opts.IsCgroup2UnifiedMode() {
 			return warnings, fmt.Errorf("runtime %s: `cgroup_writable` is only supported on cgroup v2", k)
diff --git a/internal/cri/config/config_unix.go b/internal/cri/config/config_unix.go
index 7322700b4..7d6519015 100644
--- a/internal/cri/config/config_unix.go
+++ b/internal/cri/config/config_unix.go
@@ -23,6 +23,10 @@ import (
 	"github.com/pelletier/go-toml/v2"
 )
 
+func defaultNetworkPluginBinDirs() []string {
+	return []string{"/opt/cni/bin"}
+}
+
 func DefaultImageConfig() ImageConfig {
 	return ImageConfig{
 		Snapshotter:                defaults.DefaultSnapshotter,
@@ -72,7 +76,7 @@ func DefaultRuntimeConfig() RuntimeConfig {
 
 	return RuntimeConfig{
 		CniConfig: CniConfig{
-			NetworkPluginBinDir:        "/opt/cni/bin",
+			NetworkPluginBinDirs:       defaultNetworkPluginBinDirs(),
 			NetworkPluginConfDir:       "/etc/cni/net.d",
 			NetworkPluginMaxConfNum:    1, // only one CNI plugin config file will be loaded
 			NetworkPluginSetupSerially: false,
diff --git a/internal/cri/config/config_windows.go b/internal/cri/config/config_windows.go
index 17ef08e52..789d499c6 100644
--- a/internal/cri/config/config_windows.go
+++ b/internal/cri/config/config_windows.go
@@ -23,6 +23,10 @@ import (
 	"github.com/containerd/containerd/v2/defaults"
 )
 
+func defaultNetworkPluginBinDirs() []string {
+	return []string{filepath.Join(os.Getenv("ProgramFiles"), "containerd", "cni", "bin")}
+}
+
 func DefaultImageConfig() ImageConfig {
 	return ImageConfig{
 		Snapshotter:            defaults.DefaultSnapshotter,
@@ -42,7 +46,7 @@ func DefaultImageConfig() ImageConfig {
 func DefaultRuntimeConfig() RuntimeConfig {
 	return RuntimeConfig{
 		CniConfig: CniConfig{
-			NetworkPluginBinDir:        filepath.Join(os.Getenv("ProgramFiles"), "containerd", "cni", "bin"),
+			NetworkPluginBinDirs:       defaultNetworkPluginBinDirs(),
 			NetworkPluginConfDir:       filepath.Join(os.Getenv("ProgramFiles"), "containerd", "cni", "conf"),
 			NetworkPluginMaxConfNum:    1,
 			NetworkPluginSetupSerially: false,
diff --git a/internal/cri/server/service_linux.go b/internal/cri/server/service_linux.go
index 2ba16ff12..a115e5f67 100644
--- a/internal/cri/server/service_linux.go
+++ b/internal/cri/server/service_linux.go
@@ -87,7 +87,7 @@ func (c *criService) initPlatform() (err error) {
 		i, err := cni.New(cni.WithMinNetworkCount(networkAttachCount),
 			cni.WithPluginConfDir(dir),
 			cni.WithPluginMaxConfNum(max),
-			cni.WithPluginDir([]string{c.config.NetworkPluginBinDir}))
+			cni.WithPluginDir(c.config.NetworkPluginBinDirs))
 		if err != nil {
 			return fmt.Errorf("failed to initialize cni: %w", err)
 		}
diff --git a/internal/cri/server/service_windows.go b/internal/cri/server/service_windows.go
index 47ec67ec4..1bf71413f 100644
--- a/internal/cri/server/service_windows.go
+++ b/internal/cri/server/service_windows.go
@@ -53,7 +53,7 @@ func (c *criService) initPlatform() error {
 		i, err := cni.New(cni.WithMinNetworkCount(windowsNetworkAttachCount),
 			cni.WithPluginConfDir(dir),
 			cni.WithPluginMaxConfNum(max),
-			cni.WithPluginDir([]string{c.config.NetworkPluginBinDir}))
+			cni.WithPluginDir(c.config.NetworkPluginBinDirs))
 		if err != nil {
 			return fmt.Errorf("failed to initialize cni: %w", err)
 		}
diff --git a/internal/cri/util/util.go b/internal/cri/util/util.go
index 66f532da7..e5064caad 100644
--- a/internal/cri/util/util.go
+++ b/internal/cri/util/util.go
@@ -29,24 +29,17 @@ import (
 	crilabels "github.com/containerd/containerd/v2/internal/cri/labels"
 	clabels "github.com/containerd/containerd/v2/pkg/labels"
 	"github.com/containerd/containerd/v2/pkg/namespaces"
-	"github.com/containerd/containerd/v2/pkg/timeout"
 	"github.com/containerd/log"
 )
 
-// deferCleanupTimeoutKey is used to retrieve the configurable timeout for containerd cleanup operations in defer.
-const (
-	deferCleanupTimeoutKey = "io.containerd.timeout.cri.defercleanup"
-	deferCleanupTimeout    = 1 * time.Minute
-)
-
-func init() {
-	timeout.Set(deferCleanupTimeoutKey, deferCleanupTimeout)
-}
+// deferCleanupTimeout is the default timeout for containerd cleanup operations
+// in defer.
+const deferCleanupTimeout = 1 * time.Minute
 
 // DeferContext returns a context for containerd cleanup operations in defer.
-// A configurable default timeout is applied to avoid cleanup operation pending forever.
+// A default timeout is applied to avoid cleanup operation pending forever.
 func DeferContext() (context.Context, context.CancelFunc) {
-	return context.WithTimeout(NamespacedContext(), timeout.Get(deferCleanupTimeoutKey))
+	return context.WithTimeout(NamespacedContext(), deferCleanupTimeout)
 }
 
 // NamespacedContext returns a context with kubernetes namespace set.
diff --git a/pkg/deprecation/deprecation.go b/pkg/deprecation/deprecation.go
index 603bf41f0..c672199a3 100644
--- a/pkg/deprecation/deprecation.go
+++ b/pkg/deprecation/deprecation.go
@@ -31,6 +31,8 @@ const (
 	CRIRegistryAuths Warning = Prefix + "cri-registry-auths"
 	// CRIRegistryConfigs is a warning for the use of the `configs` property
 	CRIRegistryConfigs Warning = Prefix + "cri-registry-configs"
+	// CRICNIBinDir is a warning for the use of the `bin_dir` property
+	CRICNIBinDir = Prefix + "cri-cni-bin-dir"
 	// OTLPTracingConfig is a warning for the use of the `otlp` property
 	TracingOTLPConfig Warning = Prefix + "tracing-processor-config"
 	// TracingServiceConfig is a warning for the use of the `tracing` property
@@ -52,6 +54,8 @@ var messages = map[Warning]string{
 		"Use `ImagePullSecrets` instead.",
 	CRIRegistryConfigs: "The `configs` property of `[plugins.\"io.containerd.grpc.v1.cri\".registry]` is deprecated since containerd v1.5 and will be removed in containerd v2.1." +
 		"Use `config_path` instead.",
+	CRICNIBinDir: "The `bin_dir` property of `[plugins.\"io.containerd.cri.v1.runtime\".cni`] is deprecated since containerd v2.1 and will be removed in containerd v2.2. " +
+		"Use `bin_dirs` in the same section instead.",
 
 	TracingOTLPConfig: "The `otlp` property of `[plugins.\"io.containerd.tracing.processor.v1\".otlp]` is deprecated since containerd v1.6 and will be removed in containerd v2.0." +
 		"Use OTLP environment variables instead: https://opentelemetry.io/docs/specs/otel/protocol/exporter/",
diff --git a/plugins/cri/runtime/plugin.go b/plugins/cri/runtime/plugin.go
index 6afd0ddb5..987af7c35 100644
--- a/plugins/cri/runtime/plugin.go
+++ b/plugins/cri/runtime/plugin.go
@@ -206,6 +206,9 @@ func configMigration(ctx context.Context, configVersion int, pluginConfigs map[s
 func migrateConfig(dst, src map[string]interface{}) {
 	for k, v := range src {
 		switch k {
+		case "cni":
+			// skip (handled separately below)
+			continue
 		case "containerd":
 			// skip (handled separately below)
 			continue
@@ -235,6 +238,28 @@ func migrateConfig(dst, src map[string]interface{}) {
 		}
 	}
 
+	if cniConf, ok := src["cni"].(map[string]interface{}); ok {
+		newCniConf, ok := dst["cni"].(map[string]interface{})
+		if !ok {
+			newCniConf = map[string]interface{}{}
+		}
+		for k, v := range cniConf {
+			switch k {
+			case "bin_dir":
+				// migrate `bin_dir` to `bin_dirs` only if `bin_dirs`
+				// is not already set
+				if binDirs, ok := newCniConf["bin_dirs"].([]string); !ok || len(binDirs) == 0 {
+					newCniConf["bin_dirs"] = []string{v.(string)}
+				}
+			default:
+				if _, ok := newCniConf[k]; !ok {
+					newCniConf[k] = v
+				}
+			}
+		}
+		dst["cni"] = newCniConf
+	}
+
 	// migrate cri containerd configs
 	containerdConf, ok := src["containerd"].(map[string]interface{})
 	if !ok {
diff --git a/script/setup/runc-version b/script/setup/runc-version
index a6c5252cd..212c3bf87 100644
--- a/script/setup/runc-version
+++ b/script/setup/runc-version
@@ -1,1 +1,1 @@
-v1.2.6
+v1.2.5


Test patch:
diff --git a/.github/workflows/build-test-images.yml b/.github/workflows/build-test-images.yml
index f74ef56a7..2971aecea 100644
--- a/.github/workflows/build-test-images.yml
+++ b/.github/workflows/build-test-images.yml
@@ -142,7 +142,7 @@ jobs:
           scp -i $HOME/.ssh/id_rsa ${{ env.SSH_OPTS }} azureuser@${{ env.PUBLIC_IP }}:/Users/azureuser/.docker/key.pem $HOME/.docker/key.pem
 
       - name: Login to GitHub Container Registry
-        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
+        uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
         with:
           registry: ghcr.io
           username: ${{ github.actor }}
diff --git a/internal/cri/config/config_test.go b/internal/cri/config/config_test.go
index 7a4bcd06b..896272ae6 100644
--- a/internal/cri/config/config_test.go
+++ b/internal/cri/config/config_test.go
@@ -52,6 +52,49 @@ func TestValidateConfig(t *testing.T) {
 			},
 			runtimeExpectedErr: "no corresponding runtime configured in `containerd.runtimes` for `containerd` `default_runtime_name = \"default\"",
 		},
+		"specify both cni.bin_dir and cni_bin_dirs": {
+			runtimeConfig: &RuntimeConfig{
+				ContainerdConfig: ContainerdConfig{
+					DefaultRuntimeName: RuntimeDefault,
+					Runtimes: map[string]Runtime{
+						RuntimeDefault: {},
+					},
+				},
+				CniConfig: CniConfig{
+					NetworkPluginBinDir:  "/opt/mycni/bin",
+					NetworkPluginBinDirs: []string{"/opt/mycni/bin"},
+				},
+			},
+			runtimeExpectedErr: "`cni.bin_dir` and `cni.bin_dirs` cannot be set at the same time",
+			warnings:           []deprecation.Warning{deprecation.CRICNIBinDir},
+		},
+		"cni.bin_dir, if used, is moved to cni.bin_dirs": {
+			runtimeConfig: &RuntimeConfig{
+				ContainerdConfig: ContainerdConfig{
+					DefaultRuntimeName: RuntimeDefault,
+					Runtimes: map[string]Runtime{
+						RuntimeDefault: {},
+					},
+				},
+				CniConfig: CniConfig{
+					NetworkPluginBinDir: "/opt/mycni/bin",
+				},
+			},
+			runtimeExpected: &RuntimeConfig{
+				ContainerdConfig: ContainerdConfig{
+					DefaultRuntimeName: RuntimeDefault,
+					Runtimes: map[string]Runtime{
+						RuntimeDefault: {
+							Sandboxer: string(ModePodSandbox),
+						},
+					},
+				},
+				CniConfig: CniConfig{
+					NetworkPluginBinDirs: []string{"/opt/mycni/bin"},
+				},
+			},
+			warnings: []deprecation.Warning{deprecation.CRICNIBinDir},
+		},
 
 		"deprecated auths": {
 			runtimeConfig: &RuntimeConfig{
@@ -259,6 +302,7 @@ func TestValidateConfig(t *testing.T) {
 			if test.runtimeConfig != nil {
 				w, err := ValidateRuntimeConfig(context.Background(), test.runtimeConfig)
 				if test.runtimeExpectedErr != "" {
+					assert.Error(t, err)
 					assert.Contains(t, err.Error(), test.runtimeExpectedErr)
 				} else {
 					assert.NoError(t, err)
diff --git a/plugins/cri/runtime/plugin_test.go b/plugins/cri/runtime/plugin_test.go
index 711888bd3..e902744d0 100644
--- a/plugins/cri/runtime/plugin_test.go
+++ b/plugins/cri/runtime/plugin_test.go
@@ -28,6 +28,7 @@ import (
 
 func TestCRIRuntimePluginConfigMigration(t *testing.T) {
 	runcSandboxer := "podsandbox"
+	cniBinDir := "/opt/cni/bin"
 
 	grpcCri := map[string]interface{}{
 		"enable_selinux":              true,
@@ -41,6 +42,9 @@ func TestCRIRuntimePluginConfigMigration(t *testing.T) {
 				},
 			},
 		},
+		"cni": map[string]interface{}{
+			"bin_dir": cniBinDir,
+		},
 	}
 
 	pluginConfigs := map[string]interface{}{
@@ -66,4 +70,12 @@ func TestCRIRuntimePluginConfigMigration(t *testing.T) {
 	require.NotNil(t, runc)
 	assert.Equal(t, runcSandboxer, runc["sandboxer"])
 	assert.NotContains(t, runc, "sandbox_mode")
+
+	cni, ok := runtimeConf["cni"].(map[string]interface{})
+	require.True(t, ok)
+	require.NotNil(t, cni)
+	cniBinDirs, ok := cni["bin_dirs"].([]string)
+	require.True(t, ok)
+	require.Len(t, cniBinDirs, 1)
+	require.Equal(t, cniBinDir, cniBinDirs[0])
 }