Repository: kubevirt/containerized-data-importer
Pull request #3744: Append Labels From DV to Importer Pod
Author: login: Dsanatar
Labels: name: release-note
description: Denotes a PR that will be considered when it comes time to generate release notes.
name: size/L
name: dco-signoff: yes
description: Indicates the PR's author has DCO signed all their commits.
name: lgtm
description: Indicates that a PR is ready to be merged.
name: approved
description: Indicates a PR has been approved by an approver from all required OWNERS files.
Created: 2025-05-12T19:29:36Z | Merged: 2025-05-14T22:14:02Z

Fixes #3674 

During the creation of the Importer Pod, append any labels from the PVC. This allows for labels to be passed from a Data Volume.

```release-note
Labels on DataVolumes are now copied to the importer pod
```

Linked issue #3674: Can we allow for custom labels on the importer pod, or at least add istio.io/dataplane-mode: none to support Ambient Mesh
We are running CDI `1.59.0` on Kubevirt `1.3.0`. Things have been going well until we start testing with Ambient Mesh using Istio. CDI already accounts for *sidecar-based* Istio with the annotation of `sidecar.istio.io/inject: 'false'` on the importer-pod by default, which makes the transfer bypass the Istio sidecar:

```
apiVersion: v1
kind: Pod
metadata:
  annotations:
    cdi.kubevirt.io/storage.createdByController: 'yes'
    linkerd.io/inject: disabled
    sidecar.istio.io/inject: 'false'
  labels:
    app: containerized-data-importer
    app.kubernetes.io/component: storage
    app.kubernetes.io/managed-by: cdi-controller
    cdi.kubevirt.io: importer
    prometheus.cdi.kubevirt.io: 'true'
  name: importer-prime-16bec903-1771-4e03-bad7-fa2029ccd30b
  namespace: test-vm
```

Problem is, Ambient Mesh uses a different mechanism, a **label** of `istio.io/dataplane-mode: none`. Since there is no mechanism to put this on, the importer pod launches, runs for around 2-5 min (it varies), and eventually blows up with this error:

```
I0311 18:47:50.961109       1 importer.go:103] Starting importer
I0311 18:47:50.962691       1 importer.go:173] begin import process
I0311 18:47:50.962752       1 s3-datasource.go:149] Endpoint rcd1kubes3clu1.mi.corp.xxxxx.com
I0311 18:47:50.962775       1 s3-datasource.go:153] bucket templates
I0311 18:47:50.962779       1 s3-datasource.go:154] object win2022-gold.img.gz
I0311 18:47:50.979721       1 http-datasource.go:263] Attempting to get certs from /certs/ca.pem
I0311 18:47:51.054017       1 data-processor.go:358] Calculating available size
I0311 18:47:51.055005       1 data-processor.go:366] Checking out block volume size.
I0311 18:47:51.055024       1 data-processor.go:378] Request image size not empty.
I0311 18:47:51.055038       1 data-processor.go:383] Target size 85Gi.
I0311 18:47:51.055289       1 data-processor.go:257] New phase: TransferDataFile
I0311 18:47:51.056493       1 util.go:188] Writing data...
E0311 18:50:14.174653       1 util.go:190] Unable to write file from dataReader: unexpected EOF
E0311 18:50:14.174754       1 data-processor.go:253] unexpected EOF
unable to write to file
kubevirt.io/containerized-data-importer/pkg/util.StreamDataToFile
	pkg/util/util.go:192
kubevirt.io/containerized-data-importer/pkg/importer.(*S3DataSource).TransferFile
	pkg/importer/s3-datasource.go:118
kubevirt.io/containerized-data-importer/pkg/importer.(*DataProcessor).initDefaultPhases.func4
	pkg/importer/data-processor.go:200
kubevirt.io/containerized-data-importer/pkg/importer.(*DataProcessor).ProcessDataWithPause
	pkg/importer/data-processor.go:250
kubevirt.io/containerized-data-importer/pkg/importer.(*DataProcessor).ProcessData
	pkg/importer/data-processor.go:159
main.handleImport
	cmd/cdi-importer/importer.go:179
main.main
	cmd/cdi-importer/importer.go:144
runtime.main
	GOROOT/src/runtime/proc.go:267
runtime.goexit
	src/runtime/asm_amd64.s:1650
Unable to transfer source data to target file
```

Envoy seemingly can't handle this transfer going through it, so we want to bypass it all together. 

**TL;DR** - Can we add functionality so we can add custom labels (ideal) or if not, at least add support for Ambient Istio by adding `istio.io/dataplane-mode: none`
Linked issue comments: user: login: awels
created_at: 2025-03-12T23:06:04Z
body: Hi, we made the decision to not allow any annotations on the importer pod that weren't explicitly allowed. The allowed list can be found [here](https://github.com/kubevirt/containerized-data-importer/blob/main/pkg/controller/common/util.go#L378-L384)

Feel free to open a PR to add the annotation for your networking side car. Just know we would like a test to check. If not you will have to wait until we have some time to add this. Currently buried with a lot of requests already. It should be fairly straight forward to add though.
author_association: MEMBER
user: login: jbilliau-rcd
created_at: 2025-03-17T13:53:10Z
body: Gotcha, thanks for the feedback. I'll see about trying to PR this in myself.
author_association: NONE
user: login: awels
created_at: 2025-05-05T19:27:25Z
body: I did miss that you are in fact talking about labels, and not annotations. Which should be easier to allow wholesale.
author_association: MEMBER

Diff:
diff --git a/pkg/controller/import-controller.go b/pkg/controller/import-controller.go
index 5a72627a7d..ba711b0230 100644
--- a/pkg/controller/import-controller.go
+++ b/pkg/controller/import-controller.go
@@ -879,6 +879,9 @@ func createImporterPod(ctx context.Context, log logr.Logger, client client.Clien
 
 	util.SetRecommendedLabels(pod, installerLabels, "cdi-controller")
 
+	// add any labels from pvc to the importer pod
+	util.MergeLabels(args.pvc.Labels, pod.Labels)
+
 	if err = client.Create(context.TODO(), pod); err != nil {
 		return nil, err
 	}
diff --git a/pkg/controller/populators/populator-base.go b/pkg/controller/populators/populator-base.go
index 34722921df..82589d842f 100644
--- a/pkg/controller/populators/populator-base.go
+++ b/pkg/controller/populators/populator-base.go
@@ -209,6 +209,7 @@ func (r *ReconcilerBase) createPVCPrime(pvc *corev1.PersistentVolumeClaim, sourc
 	}
 	cc.CopyAllowedAnnotations(pvc, pvcPrime)
 	util.SetRecommendedLabels(pvcPrime, r.installerLabels, "cdi-controller")
+	util.MergeLabels(pvc.Labels, pvcPrime.Labels)
 
 	// We use the populator-specific pvcModifierFunc to add required annotations
 	if updatePVCForPopulation != nil {
diff --git a/pkg/util/util.go b/pkg/util/util.go
index 708634e165..2d077efb0e 100644
--- a/pkg/util/util.go
+++ b/pkg/util/util.go
@@ -169,19 +169,6 @@ func MergeLabels(src, dest map[string]string) map[string]string {
 	return dest
 }
 
-// AppendLabels append dest labels to source (update if key has existed)
-func AppendLabels(src, dest map[string]string) map[string]string {
-	if src == nil {
-		src = map[string]string{}
-	}
-
-	for k, v := range dest {
-		src[k] = v
-	}
-
-	return src
-}
-
 // GetRecommendedInstallerLabelsFromCr returns the recommended labels to set on CDI resources
 func GetRecommendedInstallerLabelsFromCr(cr *cdiv1.CDI) map[string]string {
 	labels := map[string]string{}
diff --git a/pkg/util/util_test.go b/pkg/util/util_test.go
index 84eca04b3a..9551576749 100644
--- a/pkg/util/util_test.go
+++ b/pkg/util/util_test.go
@@ -183,3 +183,45 @@ var _ = Describe("Usable Space calculation", func() {
 		Entry("40Gi virtual size, large overhead to be 40Gi if <= 40Gi and 41Gi if > 40Gi", 40*Gi, largeOverhead),
 	)
 })
+
+var _ = Describe("Merge Labels", func() {
+
+	var (
+		someLabels, emptyLabels, existingLabels, expectedMergedLabels map[string]string
+	)
+
+	BeforeEach(func() {
+		someLabels = map[string]string{
+			"label1": "val1",
+			"label2": "val2",
+			"label3": "val3",
+		}
+		emptyLabels = make(map[string]string)
+		existingLabels = map[string]string{
+			"label4": "val4",
+			"label5": "val5",
+		}
+		expectedMergedLabels = map[string]string{
+			"label1": "val1",
+			"label2": "val2",
+			"label3": "val3",
+			"label4": "val4",
+			"label5": "val5",
+		}
+	})
+
+	DescribeTable("Should properly merge labels", func(original, merged, expected map[string]string) {
+		// copies entries from original to merged
+		MergeLabels(original, merged)
+		Expect(merged).To(HaveLen(len(expected)))
+		for key, val := range merged {
+			Expect(val).To(Equal(expected[key]))
+		}
+	},
+		Entry("original is empty", emptyLabels, someLabels, someLabels),
+		Entry("original has values", someLabels, existingLabels, expectedMergedLabels),
+		Entry("original empty, adding empty", emptyLabels, emptyLabels, emptyLabels),
+		Entry("original has values, adding empty", someLabels, emptyLabels, someLabels),
+	)
+
+})
diff --git a/tests/import_test.go b/tests/import_test.go
index 8557bca6d4..56e2445667 100644
--- a/tests/import_test.go
+++ b/tests/import_test.go
@@ -2125,6 +2125,55 @@ var _ = Describe("Containerdisk envs to PVC labels", func() {
 	)
 })
 
+var _ = Describe("Propagate DV Labels to Importer Pod", func() {
+	f := framework.NewFramework(namespacePrefix)
+
+	const (
+		testKubevirtKey    = "test.kubevirt.io/test"
+		testKubevirtValue  = "true"
+		testNonKubevirtKey = "testLabel"
+		testNonKubevirtVal = "none"
+	)
+
+	DescribeTable("Import pod should inherit any labels from Data Volume", func(usePopulator string) {
+
+		dataVolume := utils.NewDataVolumeWithHTTPImport("label-test", "100Mi", fmt.Sprintf(utils.TinyCoreIsoURL, f.CdiInstallNs))
+		dataVolume.Annotations[controller.AnnImmediateBinding] = "true"
+		dataVolume.Annotations[controller.AnnPodRetainAfterCompletion] = "true"
+		dataVolume.Annotations[controller.AnnUsePopulator] = usePopulator
+
+		dataVolume.Labels = map[string]string{
+			testKubevirtKey:    testKubevirtValue,
+			testNonKubevirtKey: testNonKubevirtVal,
+		}
+
+		By(fmt.Sprintf("Create new datavolume %s", dataVolume.Name))
+		dataVolume, err := utils.CreateDataVolumeFromDefinition(f.CdiClient, f.Namespace.Name, dataVolume)
+		Expect(err).ToNot(HaveOccurred())
+
+		By("Verify pvc was created")
+		_, err = utils.WaitForPVC(f.K8sClient, dataVolume.Namespace, dataVolume.Name)
+		Expect(err).ToNot(HaveOccurred())
+
+		By("Wait for import to be completed")
+		err = utils.WaitForDataVolumePhase(f, dataVolume.Namespace, cdiv1.Succeeded, dataVolume.Name)
+		Expect(err).ToNot(HaveOccurred(), "Datavolume not in phase succeeded in time")
+
+		By("Find importer pod")
+		importer, err := utils.FindPodByPrefix(f.K8sClient, dataVolume.Namespace, common.ImporterPodName, common.CDILabelSelector)
+		Expect(err).ToNot(HaveOccurred())
+
+		By("Check labels were appended")
+		importLabels := importer.GetLabels()
+		Expect(importLabels).Should(HaveKeyWithValue(testKubevirtKey, testKubevirtValue))
+		Expect(importLabels).Should(HaveKeyWithValue(testNonKubevirtKey, testNonKubevirtVal))
+
+	},
+		Entry("With Populators", "true"),
+		Entry("Without Populators", "false"),
+	)
+})
+
 var _ = Describe("pull image failure", func() {
 	var (
 		f = framework.NewFramework(namespacePrefix)


Reviews:
user: login: awels
body: Looks like the failures are because the labels are not being applied to the importer pod. That is what the error is indicating.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-13T15:42:44Z
user: login: awels
body: /approve
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-14T15:53:07Z

Review comments:
diff_hunk: @@ -2125,6 +2125,51 @@ var _ = Describe("Containerdisk envs to PVC labels", func() {
 	)
 })
 
+var _ = Describe("Propagate DV Labels to Importer Pod", func() {
+	f := framework.NewFramework(namespacePrefix)
+
+	const (
+		testKubevirtKey    = "test.kubevirt.io/test"
+		testKubevirtValue  = "true"
+		testNonKubevirtKey = "testLabel"
+		testNonKubevirtVal = "none"
+	)
+
+	It("Import pod should inherit any labels from Data Volume", func() {
+
+		dataVolume := utils.NewDataVolumeWithHTTPImport("label-test", "100Mi", fmt.Sprintf(utils.TinyCoreIsoURL, f.CdiInstallNs))
+		dataVolume.Annotations[controller.AnnImmediateBinding] = "true"
+		dataVolume.Annotations[controller.AnnPodRetainAfterCompletion] = "true"
+
+		dataVolume.ObjectMeta.Labels = map[string]string{
path: tests/import_test.go
user: login: awels
body: You don't have to specify ObjectMeta here, so just `dataVolume.Labels` will do.
created_at: 2025-05-13T15:18:55Z
author_association: MEMBER
diff_hunk: @@ -183,3 +183,39 @@ var _ = Describe("Usable Space calculation", func() {
 		Entry("40Gi virtual size, large overhead to be 40Gi if <= 40Gi and 41Gi if > 40Gi", 40*Gi, largeOverhead),
 	)
 })
+
+var _ = Describe("Append Labels", func() {
+
+	someLabels := map[string]string{
path: pkg/util/util_test.go
user: login: awels
body: I would make this a `DescribeTable` since we are just testing various permutations of inputs, and comparing it against an output. Something like this:
```go
var (
  someLabels, emtpyLabels, existingLabels, expectedMergedLabels map[string]string

BeforeEach(func() {
  someLabels = map[string]string{
    "label1": "val1",
    "label2": "val2",
    "label3": "val3",
  }
  emptyLabels = make(map[string]string)
  existingLabels = map[string]string{
    "label4": "val4",
    "label5": "val5",
  }
  expectedMergedLabels = map[string]string{
    "label1": "val1",
    "label2": "val2",
    "label3": "val3",
    "label4": "val4",
    "label5": "val5",
  }
})
DescribeTable("should properly merge labels", func(original, merge, expected map[string]string) {
  MergeLabels(original, merge)
  Expect(original).To(HaveLen(len(expected)))
  for k,v := range original {
    Expect(expected[k]).To(Equal(original[k]))
  }
}, 
Entry("original empty", someLabels, emptyLabels, someLabels),
Entry("original has values", someLabels, existingLabels, expectedMergedLabels),
Entry("original empty, adding empty", emptyLabels, emptyLabels, emptyLabels),
Entry("original has values, adding empty", emptyLabels, existingLabels, existingLabels),
)
```
This allows me to try a variety of permutations with a single check. Also by using the BeforeEach, I can reset the variables to a know good value before each time an entry is invoked.
created_at: 2025-05-13T15:36:46Z
author_association: MEMBER

PR comments:
user: login: kubevirt-bot
created_at: 2025-05-14T15:53:44Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubevirt/containerized-data-importer/pull/3744#pullrequestreview-2840732462" title="Approved">awels</a>*

The full list of commands accepted by this bot can be found [here](https://prow.ci.kubevirt.io/command-help?repo=kubevirt%2Fcontainerized-data-importer).

The pull request process is described [here](https://github.com/kubevirt/community/blob/main/docs/add-merge-automation-to-your-repository.md#example-workflow)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubevirt/containerized-data-importer/blob/main/OWNERS)~~ [awels]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: CONTRIBUTOR
user: login: awels
created_at: 2025-05-14T19:16:52Z
body: /test pull-cdi-generate-verify
author_association: MEMBER