Repository: kubernetes-sigs/node-feature-discovery
Pull request #431 (2021-02-03 21:21:11)

nfd-master: implement --instance flag
This can be used to help running multiple parallel NFD deployments in
the same cluster. The flag changes the node annotation namespace to
<instance>.nfd.node.kubernetes.io allowing different nfd-master intances
to store metadata in separate annotations.

Related to #427 

k8s-ci-robot: Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with `/test all`
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/node-feature-discovery/pull/431#" title="Author self-approved">marquiz</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fnode-feature-discovery).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/OWNERS)~~ [marquiz]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
e0ne: It doesn't fixes #427. Adding new annotations works well, but different NDF instances will override each other labels. It means any software which relates on standard NFD behaviour and labels won't work good in case when two or more NFD are deployed.
marquiz: > It doesn't fixes #427. Adding new annotations works well, but different NDF instances will override each other labels. It means any software which relates on standard NFD behaviour and labels won't work good in case when two or more NFD are deployed.

My point is that you need to configure the "label separation" somewhere, either master or worker side. Does it really matter if it would be on worker side? Worker side is much more flexible, e.g. you can directly disable all unneeded feature sources.
e0ne: @marquiz is it a supported configuration to have 2+ workers on the same node with different configuration?
marquiz: > override each other labels

What kind of label overwrite are you concerned about? If two instances write cpuid.XYZ or kernel version it shouldn't even matter as the value from both sources should be the same. When running multiple instances of NFD in parallel there is really no reliable way prevent overlap, i.e. prevent different instances modifying the same labels. An incorrect deployment/configuration of one NFD instance will ruin everything.
marquiz: > @marquiz is it a supported configuration to have 2+ workers on the same node with different configuration?

Yes, sure. They are separate instances (pods) so they may have different configuration. The "configuration" is basically the command line flags plus the configuration file. Configuration file is typically taken from a config map. Just use different configmap for different instances (e.g. run them in separate k8s namespaces)
e0ne: E..g.:
instance1: --label-whitelist=pci-0300_102b.present"
instance2: --label-whitelist=pci-8086.sriov.capable"

As a result, I would like to have a list of labels which contains both pci-0300_102b and pci-8086.sriov features discovered
e0ne: In other words, master should merge labels provided by different workers
zvonkok: @e0ne Why would different workers have different labels? Two instances of NFD would be running in e.g. different namespaces, not on different nodes. 
zvonkok: Each worker would discover the very same features. @e0ne Are you planning to run the second instance of NFD on a subset of available nodes? Usually NFD is deployed to all workers. 
e0ne: @zvonkok here is an example how we deploy NFD: https://github.com/kubernetes-sigs/node-feature-discovery/issues/427#issuecomment-772554744
zvonkok: Ah got it. You have different configurations for each instance of NFD. 
e0ne: > Ah got it. You have different configurations for each instance of NFD.

Exactly, that's why we need several instances of NDF
marquiz: > In other words, master should merge labels provided by different workers

As I commented in #427, this totally different and orthogonal AFAICS. Could you open a separate issue about this where we could discuss it?
e0ne: we can add 'instance' prefix to labels, not for annotations and it should work in a such way: each NFD instance will override labels with 'instance' prefix and can deploy a worker without prefix as a default
marquiz: > we can add 'instance' prefix to labels, not for annotations and it should work in a such way: each NFD instance will override labels with 'instance' prefix and can deploy a worker without prefix as a default

This sounds like not a very good idea and would just add confusion, I think.

I still fail to see what you'd gain with this. If two deployments would create the same label (name) with a different value this is clearly a configuration error. They should use a different label name, e.g. by using a dedicated label ns like `vendor-1.com/feature-x` and `vendor-2.com/feature-x`. If you're talking about the "built-in" feature labels like `cpu-cpuid.XYZ` or `pci-<device>.present` etc the value really should not be different between deployments
k8s-ci-robot: @marquiz: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-node-feature-discovery-build-image | ab8730a0438c4e80c3797eb1006eb05411923161 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_node-feature-discovery/431/pull-node-feature-discovery-build-image/1357417424688779264) | `/test pull-node-feature-discovery-build-image`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=node-feature-discovery&pr=431). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Amarquiz). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/node-feature-discovery/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/node-feature-discovery/pull/431#" title="Author self-approved">marquiz</a>*, *<a href="https://github.com/kubernetes-sigs/node-feature-discovery/pull/431#pullrequestreview-585844624" title="Approved">moshe010</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fnode-feature-discovery).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/OWNERS)~~ [marquiz]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
marquiz: > Thanks for working on this. We tested it in it solves our problem :)

Thanks for testing. I haven't found any problems in my own test setup. Just waiting for a review....

What I really want to have is an e2e-test for this but I think that could be addressed in a separate PR
moshe010: @zvonkok, Can you please review this PR? 
marquiz: @moshe010 you can also take a look at the changes and provide comments, that would be much appreciated 😉 
marquiz: Thanks for the review @e0ne and @adrianchiris ! Addressed the comments in the topmost patch

I want to squash the fixes in the other patches before merging
/hold
marquiz: > first commits are refactors which are not necessarily needed for support for instance flag, but are sensible and simplifies code.

True, the refactors are not strictly necessary, but without them the code would far uglier as the annotation ns mangling messes things up. I tried and didn't like it 😄  I didn't see much point in putting the refactors in a separate PR, but, could be easily done if people like it better that way
adrianchiris: > True, the refactors are not strictly necessary, but without them the code would far uglier as the annotation ns mangling messes things up. I tried and didn't like it 😄 I didn't see much point in putting the refactors in a separate PR, but, could be easily done if people like it better that way

Im totally fine with having in the same PR.
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/node-feature-discovery/pull/431#pullrequestreview-587509076" title="Approved">adrianchiris</a>*, *<a href="https://github.com/kubernetes-sigs/node-feature-discovery/pull/431#" title="Author self-approved">marquiz</a>*, *<a href="https://github.com/kubernetes-sigs/node-feature-discovery/pull/431#pullrequestreview-585844624" title="Approved">moshe010</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fnode-feature-discovery).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/OWNERS)~~ [marquiz]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->

Patch:
diff --git a/cmd/nfd-master/main.go b/cmd/nfd-master/main.go
index 5247477d0..9f66a37c3 100644
--- a/cmd/nfd-master/main.go
+++ b/cmd/nfd-master/main.go
@@ -66,7 +66,7 @@ func argsParse(argv []string) (master.Args, error) {
   %s [--prune] [--no-publish] [--label-whitelist=<pattern>] [--port=<port>]
      [--ca-file=<path>] [--cert-file=<path>] [--key-file=<path>]
      [--verify-node-name] [--extra-label-ns=<list>] [--resource-labels=<list>]
-     [--kubeconfig=<path>]
+     [--kubeconfig=<path>] [--instance=<name>]
   %s -h | --help
   %s --version
 
@@ -75,6 +75,9 @@ func argsParse(argv []string) (master.Args, error) {
   --version                       Output version and exit.
   --prune                         Prune all NFD related attributes from all nodes
                                   of the cluster and exit.
+  --instance=<name>               Instance name. Used to separate annotation
+                                  namespaces for multiple parallel deployments.
+                                  [Default: ]
   --kubeconfig=<path>             Kubeconfig to use [Default: ]
                                   of the cluster and exit.
   --port=<port>                   Port on which to listen for connections.
@@ -109,6 +112,12 @@ func argsParse(argv []string) (master.Args, error) {
 
 	// Parse argument values as usable types.
 	var err error
+	args.Instance = arguments["--instance"].(string)
+	if ok, _ := regexp.MatchString(`^([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]$`, args.Instance); args.Instance != "" && !ok {
+		return args, fmt.Errorf("invalid --instance %q: instance name "+
+			"must start and end with an alphanumeric character and may only contain "+
+			"alphanumerics, `-`, `_` or `.`", args.Instance)
+	}
 	args.CaFile = arguments["--ca-file"].(string)
 	args.CertFile = arguments["--cert-file"].(string)
 	args.KeyFile = arguments["--key-file"].(string)
diff --git a/docs/advanced/master-commandline-reference.md b/docs/advanced/master-commandline-reference.md
index 22aad46c0..58089b0de 100644
--- a/docs/advanced/master-commandline-reference.md
+++ b/docs/advanced/master-commandline-reference.md
@@ -48,6 +48,22 @@ Example:
 nfd-master --port=443
 ```
 
+### --instance
+
+The `--instance` flag makes it possible to run multiple NFD deployments in
+parallel. In practice, it separates the node annotations between deployments so
+that each of them can store metadata independently. The instance name must
+start and end with an alphanumeric character and may only contain alphanumeric
+characters, `-`, `_` or `.`.
+
+Default: *empty*
+
+Example:
+
+```bash
+nfd-master --instance=network
+```
+
 ### --ca-file
 
 The `--ca-file` is one of the three flags (together with `--cert-file` and
diff --git a/docs/get-started/introduction.md b/docs/get-started/introduction.md
index cd2fe1a47..cc75a0568 100644
--- a/docs/get-started/introduction.md
+++ b/docs/get-started/introduction.md
@@ -80,12 +80,16 @@ An overview of the default feature labels:
 
 NFD also annotates nodes it is running on:
 
-| Annotation                                | Description
-| ----------------------------------------- | -----------
-| nfd.node.kubernetes.io/master.version     | Version of the nfd-master instance running on the node. Informative use only.
-| nfd.node.kubernetes.io/worker.version     | Version of the nfd-worker instance running on the node. Informative use only.
-| nfd.node.kubernetes.io/feature-labels     | Comma-separated list of node labels managed by NFD. NFD uses this internally so must not be edited by users.
-| nfd.node.kubernetes.io/extended-resources | Comma-separated list of node extended resources managed by NFD. NFD uses this internally so must not be edited by users.
+| Annotation                                                   | Description
+| ------------------------------------------------------------ | -----------
+| [&lt;instance&gt;.]nfd.node.kubernetes.io/master.version     | Version of the nfd-master instance running on the node. Informative use only.
+| [&lt;instance&gt;.]nfd.node.kubernetes.io/worker.version     | Version of the nfd-worker instance running on the node. Informative use only.
+| [&lt;instance&gt;.]nfd.node.kubernetes.io/feature-labels     | Comma-separated list of node labels managed by NFD. NFD uses this internally so must not be edited by users.
+| [&lt;instance&gt;.]nfd.node.kubernetes.io/extended-resources | Comma-separated list of node extended resources managed by NFD. NFD uses this internally so must not be edited by users.
+
+NOTE: the [`--instance`](../advanced/master-commandline-reference.md#instance)
+command line flag affects the annotation names
 
 Unapplicable annotations are not created, i.e. for example master.version is only created on nodes running nfd-master.
 
+
diff --git a/pkg/nfd-master/nfd-master.go b/pkg/nfd-master/nfd-master.go
index 6196bb843..6a3ac39c5 100644
--- a/pkg/nfd-master/nfd-master.go
+++ b/pkg/nfd-master/nfd-master.go
@@ -45,21 +45,20 @@ const (
 	// Namespace for feature labels
 	LabelNs = "feature.node.kubernetes.io"
 
-	// Namespace for all NFD-related annotations
-	AnnotationNs = "nfd.node.kubernetes.io"
+	// Base namespace for all NFD-related annotations
+	AnnotationNsBase = "nfd.node.kubernetes.io"
 
 	// NFD Annotations
-	extendedResourceAnnotation = AnnotationNs + "/extended-resources"
-	featureLabelAnnotation     = AnnotationNs + "/feature-labels"
-	masterVersionAnnotation    = AnnotationNs + "/master.version"
-	workerVersionAnnotation    = AnnotationNs + "/worker.version"
+	extendedResourceAnnotation = "extended-resources"
+	featureLabelAnnotation     = "feature-labels"
+	masterVersionAnnotation    = "master.version"
+	workerVersionAnnotation    = "worker.version"
 )
 
 // package loggers
 var (
 	stdoutLogger = log.New(os.Stdout, "", log.LstdFlags)
 	stderrLogger = log.New(os.Stderr, "", log.LstdFlags)
-	nodeName     = os.Getenv("NODE_NAME")
 )
 
 // Labels are a Kubernetes representation of discovered features.
@@ -76,6 +75,7 @@ type Args struct {
 	CaFile         string
 	CertFile       string
 	ExtraLabelNs   map[string]struct{}
+	Instance       string
 	KeyFile        string
 	Kubeconfig     string
 	LabelWhiteList *regexp.Regexp
@@ -93,15 +93,26 @@ type NfdMaster interface {
 }
 
 type nfdMaster struct {
-	args      Args
-	server    *grpc.Server
-	ready     chan bool
-	apihelper apihelper.APIHelpers
+	args         Args
+	nodeName     string
+	annotationNs string
+	server       *grpc.Server
+	ready        chan bool
+	apihelper    apihelper.APIHelpers
 }
 
 // Create new NfdMaster server instance.
 func NewNfdMaster(args Args) (NfdMaster, error) {
-	nfd := &nfdMaster{args: args, ready: make(chan bool, 1)}
+	nfd := &nfdMaster{args: args,
+		nodeName: os.Getenv("NODE_NAME"),
+		ready:    make(chan bool, 1),
+	}
+
+	if args.Instance == "" {
+		nfd.annotationNs = AnnotationNsBase
+	} else {
+		nfd.annotationNs = args.Instance + "." + AnnotationNsBase
+	}
 
 	// Check TLS related args
 	if args.CertFile != "" || args.KeyFile != "" || args.CaFile != "" {
@@ -126,14 +137,17 @@ func NewNfdMaster(args Args) (NfdMaster, error) {
 // is called.
 func (m *nfdMaster) Run() error {
 	stdoutLogger.Printf("Node Feature Discovery Master %s", version.Get())
-	stdoutLogger.Printf("NodeName: '%s'", nodeName)
+	if m.args.Instance != "" {
+		stdoutLogger.Printf("Master instance: '%s'", m.args.Instance)
+	}
+	stdoutLogger.Printf("NodeName: '%s'", m.nodeName)
 
 	if m.args.Prune {
 		return m.prune()
 	}
 
 	if !m.args.NoPublish {
-		err := updateMasterNode(m.apihelper)
+		err := m.updateMasterNode()
 		if err != nil {
 			return fmt.Errorf("failed to update master node: %v", err)
 		}
@@ -175,7 +189,7 @@ func (m *nfdMaster) Run() error {
 		serverOpts = append(serverOpts, grpc.Creds(credentials.NewTLS(tlsConfig)))
 	}
 	m.server = grpc.NewServer(serverOpts...)
-	pb.RegisterLabelerServer(m.server, &labelerServer{args: m.args, apiHelper: m.apihelper})
+	pb.RegisterLabelerServer(m.server, m)
 	stdoutLogger.Printf("gRPC server serving on port: %d", m.args.Port)
 	return m.server.Serve(lis)
 }
@@ -216,7 +230,7 @@ func (m *nfdMaster) prune() error {
 		stdoutLogger.Printf("pruning node %q...", node.Name)
 
 		// Prune labels and extended resources
-		err := updateNodeFeatures(m.apihelper, node.Name, Labels{}, Annotations{}, ExtendedResources{})
+		err := m.updateNodeFeatures(node.Name, Labels{}, Annotations{}, ExtendedResources{})
 		if err != nil {
 			return fmt.Errorf("failed to prune labels from node %q: %v", node.Name, err)
 		}
@@ -227,7 +241,7 @@ func (m *nfdMaster) prune() error {
 			return err
 		}
 		for a := range node.Annotations {
-			if strings.HasPrefix(a, AnnotationNs) {
+			if strings.HasPrefix(a, m.annotationNs) {
 				delete(node.Annotations, a)
 			}
 		}
@@ -241,19 +255,22 @@ func (m *nfdMaster) prune() error {
 }
 
 // Advertise NFD master information
-func updateMasterNode(helper apihelper.APIHelpers) error {
-	cli, err := helper.GetClient()
+func (m *nfdMaster) updateMasterNode() error {
+	cli, err := m.apihelper.GetClient()
 	if err != nil {
 		return err
 	}
-	node, err := helper.GetNode(cli, nodeName)
+	node, err := m.apihelper.GetNode(cli, m.nodeName)
 	if err != nil {
 		return err
 	}
 
 	// Advertise NFD version as an annotation
-	p := createPatches(nil, node.Annotations, Annotations{masterVersionAnnotation: version.Get()}, "/metadata/annotations")
-	err = helper.PatchNode(cli, node.Name, p)
+	p := createPatches(nil,
+		node.Annotations,
+		Annotations{m.annotationName(masterVersionAnnotation): version.Get()},
+		"/metadata/annotations")
+	err = m.apihelper.PatchNode(cli, node.Name, p)
 	if err != nil {
 		stderrLogger.Printf("failed to patch node annotations: %v", err)
 		return err
@@ -310,15 +327,9 @@ func filterFeatureLabels(labels Labels, extraLabelNs map[string]struct{}, labelW
 	return outLabels, extendedResources
 }
 
-// Implement LabelerServer
-type labelerServer struct {
-	args      Args
-	apiHelper apihelper.APIHelpers
-}
-
-// Service SetLabels
-func (s *labelerServer) SetLabels(c context.Context, r *pb.SetLabelsRequest) (*pb.SetLabelsReply, error) {
-	if s.args.VerifyNodeName {
+// SetLabels implements LabelerServer
+func (m *nfdMaster) SetLabels(c context.Context, r *pb.SetLabelsRequest) (*pb.SetLabelsReply, error) {
+	if m.args.VerifyNodeName {
 		// Client authorization.
 		// Check that the node name matches the CN from the TLS cert
 		client, ok := peer.FromContext(c)
@@ -343,13 +354,13 @@ func (s *labelerServer) SetLabels(c context.Context, r *pb.SetLabelsRequest) (*p
 	}
 	stdoutLogger.Printf("REQUEST Node: %s NFD-version: %s Labels: %s", r.NodeName, r.NfdVersion, r.Labels)
 
-	labels, extendedResources := filterFeatureLabels(r.Labels, s.args.ExtraLabelNs, s.args.LabelWhiteList, s.args.ResourceLabels)
+	labels, extendedResources := filterFeatureLabels(r.Labels, m.args.ExtraLabelNs, m.args.LabelWhiteList, m.args.ResourceLabels)
 
-	if !s.args.NoPublish {
+	if !m.args.NoPublish {
 		// Advertise NFD worker version as an annotation
-		annotations := Annotations{workerVersionAnnotation: r.NfdVersion}
+		annotations := Annotations{m.annotationName(workerVersionAnnotation): r.NfdVersion}
 
-		err := updateNodeFeatures(s.apiHelper, r.NodeName, labels, annotations, extendedResources)
+		err := m.updateNodeFeatures(r.NodeName, labels, annotations, extendedResources)
 		if err != nil {
 			stderrLogger.Printf("failed to advertise labels: %s", err.Error())
 			return &pb.SetLabelsReply{}, err
@@ -361,14 +372,14 @@ func (s *labelerServer) SetLabels(c context.Context, r *pb.SetLabelsRequest) (*p
 // updateNodeFeatures ensures the Kubernetes node object is up to date,
 // creating new labels and extended resources where necessary and removing
 // outdated ones. Also updates the corresponding annotations.
-func updateNodeFeatures(helper apihelper.APIHelpers, nodeName string, labels Labels, annotations Annotations, extendedResources ExtendedResources) error {
-	cli, err := helper.GetClient()
+func (m *nfdMaster) updateNodeFeatures(nodeName string, labels Labels, annotations Annotations, extendedResources ExtendedResources) error {
+	cli, err := m.apihelper.GetClient()
 	if err != nil {
 		return err
 	}
 
 	// Get the worker node object
-	node, err := helper.GetNode(cli, nodeName)
+	node, err := m.apihelper.GetNode(cli, nodeName)
 	if err != nil {
 		return err
 	}
@@ -380,7 +391,7 @@ func updateNodeFeatures(helper apihelper.APIHelpers, nodeName string, labels Lab
 		labelKeys = append(labelKeys, strings.TrimPrefix(key, LabelNs+"/"))
 	}
 	sort.Strings(labelKeys)
-	annotations[featureLabelAnnotation] = strings.Join(labelKeys, ",")
+	annotations[m.annotationName(featureLabelAnnotation)] = strings.Join(labelKeys, ",")
 
 	// Store names of extended resources in an annotation
 	extendedResourceKeys := make([]string, 0, len(extendedResources))
@@ -389,10 +400,10 @@ func updateNodeFeatures(helper apihelper.APIHelpers, nodeName string, labels Lab
 		extendedResourceKeys = append(extendedResourceKeys, strings.TrimPrefix(key, LabelNs+"/"))
 	}
 	sort.Strings(extendedResourceKeys)
-	annotations[extendedResourceAnnotation] = strings.Join(extendedResourceKeys, ",")
+	annotations[m.annotationName(extendedResourceAnnotation)] = strings.Join(extendedResourceKeys, ",")
 
 	// Create JSON patches for changes in labels and annotations
-	oldLabels := stringToNsNames(node.Annotations[featureLabelAnnotation], LabelNs)
+	oldLabels := stringToNsNames(node.Annotations[m.annotationName(featureLabelAnnotation)], LabelNs)
 	patches := createPatches(oldLabels, node.Labels, labels, "/metadata/labels")
 	patches = append(patches, createPatches(nil, node.Annotations, annotations, "/metadata/annotations")...)
 
@@ -401,15 +412,15 @@ func updateNodeFeatures(helper apihelper.APIHelpers, nodeName string, labels Lab
 	patches = append(patches, removeLabelsWithPrefix(node, "node.alpha.kubernetes-incubator.io/node-feature-discovery")...)
 
 	// Patch the node object in the apiserver
-	err = helper.PatchNode(cli, node.Name, patches)
+	err = m.apihelper.PatchNode(cli, node.Name, patches)
 	if err != nil {
 		stderrLogger.Printf("error while patching node object: %s", err.Error())
 		return err
 	}
 
 	// patch node status with extended resource changes
-	patches = createExtendedResourcePatches(node, extendedResources)
-	err = helper.PatchNodeStatus(cli, node.Name, patches)
+	patches = m.createExtendedResourcePatches(node, extendedResources)
+	err = m.apihelper.PatchNodeStatus(cli, node.Name, patches)
 	if err != nil {
 		stderrLogger.Printf("error while patching extended resources: %s", err.Error())
 		return err
@@ -418,6 +429,10 @@ func updateNodeFeatures(helper apihelper.APIHelpers, nodeName string, labels Lab
 	return err
 }
 
+func (m *nfdMaster) annotationName(name string) string {
+	return path.Join(m.annotationNs, name)
+}
+
 // Remove any labels having the given prefix
 func removeLabelsWithPrefix(n *api.Node, search string) []apihelper.JsonPatch {
 	var p []apihelper.JsonPatch
@@ -460,11 +475,11 @@ func createPatches(removeKeys []string, oldItems map[string]string, newItems map
 
 // createExtendedResourcePatches returns a slice of operations to perform on
 // the node status
-func createExtendedResourcePatches(n *api.Node, extendedResources ExtendedResources) []apihelper.JsonPatch {
+func (m *nfdMaster) createExtendedResourcePatches(n *api.Node, extendedResources ExtendedResources) []apihelper.JsonPatch {
 	patches := []apihelper.JsonPatch{}
 
 	// Form a list of namespaced resource names managed by us
-	oldResources := stringToNsNames(n.Annotations[extendedResourceAnnotation], LabelNs)
+	oldResources := stringToNsNames(n.Annotations[m.annotationName(extendedResourceAnnotation)], LabelNs)
 
 	// figure out which resources to remove
 	for _, resource := range oldResources {


Test patch:
diff --git a/pkg/nfd-master/nfd-master-internal_test.go b/pkg/nfd-master/nfd-master-internal_test.go
index fbc7e3a70..45a1d239e 100644
--- a/pkg/nfd-master/nfd-master-internal_test.go
+++ b/pkg/nfd-master/nfd-master-internal_test.go
@@ -17,6 +17,7 @@ limitations under the License.
 package nfdmaster
 
 import (
+	"path"
 	"regexp"
 	"sort"
 	"strings"
@@ -41,10 +42,6 @@ const (
 	mockNodeName = "mock-node"
 )
 
-func init() {
-	nodeName = mockNodeName
-}
-
 func newMockNode() *api.Node {
 	n := api.Node{}
 	n.Name = mockNodeName
@@ -54,6 +51,15 @@ func newMockNode() *api.Node {
 	return &n
 }
 
+func newMockMaster(apihelper apihelper.APIHelpers) *nfdMaster {
+	return &nfdMaster{
+		nodeName:     mockNodeName,
+		annotationNs: AnnotationNsBase,
+		args:         Args{LabelWhiteList: regexp.MustCompile("")},
+		apihelper:    apihelper,
+	}
+}
+
 func TestUpdateNodeFeatures(t *testing.T) {
 	Convey("When I update the node using fake client", t, func() {
 		fakeFeatureLabels := map[string]string{LabelNs + "/source-feature.1": "1", LabelNs + "/source-feature.2": "2", LabelNs + "/source-feature.3": "val3"}
@@ -73,17 +79,18 @@ func TestUpdateNodeFeatures(t *testing.T) {
 		sort.Strings(fakeExtResourceNames)
 
 		mockAPIHelper := new(apihelper.MockAPIHelpers)
+		mockMaster := newMockMaster(mockAPIHelper)
 		mockClient := &k8sclient.Clientset{}
 		// Mock node with old features
 		mockNode := newMockNode()
 		mockNode.Labels[LabelNs+"/old-feature"] = "old-value"
-		mockNode.Annotations[AnnotationNs+"/feature-labels"] = "old-feature"
+		mockNode.Annotations[AnnotationNsBase+"/feature-labels"] = "old-feature"
 
 		Convey("When I successfully update the node with feature labels", func() {
 			// Create a list of expected node metadata patches
 			metadataPatches := []apihelper.JsonPatch{
-				apihelper.NewJsonPatch("replace", "/metadata/annotations", AnnotationNs+"/feature-labels", strings.Join(fakeFeatureLabelNames, ",")),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", AnnotationNs+"/extended-resources", strings.Join(fakeExtResourceNames, ",")),
+				apihelper.NewJsonPatch("replace", "/metadata/annotations", AnnotationNsBase+"/feature-labels", strings.Join(fakeFeatureLabelNames, ",")),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", AnnotationNsBase+"/extended-resources", strings.Join(fakeExtResourceNames, ",")),
 				apihelper.NewJsonPatch("remove", "/metadata/labels", LabelNs+"/old-feature", ""),
 			}
 			for k, v := range fakeFeatureLabels {
@@ -103,7 +110,7 @@ func TestUpdateNodeFeatures(t *testing.T) {
 			mockAPIHelper.On("GetNode", mockClient, mockNodeName).Return(mockNode, nil).Once()
 			mockAPIHelper.On("PatchNode", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(metadataPatches))).Return(nil)
 			mockAPIHelper.On("PatchNodeStatus", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(statusPatches))).Return(nil)
-			err := updateNodeFeatures(mockAPIHelper, mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
+			err := mockMaster.updateNodeFeatures(mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
 
 			Convey("Error is nil", func() {
 				So(err, ShouldBeNil)
@@ -113,7 +120,7 @@ func TestUpdateNodeFeatures(t *testing.T) {
 		Convey("When I fail to update the node with feature labels", func() {
 			expectedError := errors.New("fake error")
 			mockAPIHelper.On("GetClient").Return(nil, expectedError)
-			err := updateNodeFeatures(mockAPIHelper, mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
+			err := mockMaster.updateNodeFeatures(mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
 
 			Convey("Error is produced", func() {
 				So(err, ShouldEqual, expectedError)
@@ -123,7 +130,7 @@ func TestUpdateNodeFeatures(t *testing.T) {
 		Convey("When I fail to get a mock client while updating feature labels", func() {
 			expectedError := errors.New("fake error")
 			mockAPIHelper.On("GetClient").Return(nil, expectedError)
-			err := updateNodeFeatures(mockAPIHelper, mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
+			err := mockMaster.updateNodeFeatures(mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
 
 			Convey("Error is produced", func() {
 				So(err, ShouldEqual, expectedError)
@@ -134,7 +141,7 @@ func TestUpdateNodeFeatures(t *testing.T) {
 			expectedError := errors.New("fake error")
 			mockAPIHelper.On("GetClient").Return(mockClient, nil)
 			mockAPIHelper.On("GetNode", mockClient, mockNodeName).Return(nil, expectedError).Once()
-			err := updateNodeFeatures(mockAPIHelper, mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
+			err := mockMaster.updateNodeFeatures(mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
 
 			Convey("Error is produced", func() {
 				So(err, ShouldEqual, expectedError)
@@ -146,7 +153,7 @@ func TestUpdateNodeFeatures(t *testing.T) {
 			mockAPIHelper.On("GetClient").Return(mockClient, nil)
 			mockAPIHelper.On("GetNode", mockClient, mockNodeName).Return(mockNode, nil).Once()
 			mockAPIHelper.On("PatchNode", mockClient, mockNodeName, mock.Anything).Return(expectedError).Once()
-			err := updateNodeFeatures(mockAPIHelper, mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
+			err := mockMaster.updateNodeFeatures(mockNodeName, fakeFeatureLabels, fakeAnnotations, fakeExtResources)
 
 			Convey("Error is produced", func() {
 				So(err, ShouldEqual, expectedError)
@@ -159,15 +166,16 @@ func TestUpdateNodeFeatures(t *testing.T) {
 func TestUpdateMasterNode(t *testing.T) {
 	Convey("When updating the nfd-master node", t, func() {
 		mockHelper := &apihelper.MockAPIHelpers{}
+		mockMaster := newMockMaster(mockHelper)
 		mockClient := &k8sclient.Clientset{}
 		mockNode := newMockNode()
 		Convey("When update operation succeeds", func() {
 			expectedPatches := []apihelper.JsonPatch{
-				apihelper.NewJsonPatch("add", "/metadata/annotations", AnnotationNs+"/master.version", version.Get())}
+				apihelper.NewJsonPatch("add", "/metadata/annotations", AnnotationNsBase+"/master.version", version.Get())}
 			mockHelper.On("GetClient").Return(mockClient, nil)
 			mockHelper.On("GetNode", mockClient, mockNodeName).Return(mockNode, nil)
 			mockHelper.On("PatchNode", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedPatches))).Return(nil)
-			err := updateMasterNode(mockHelper)
+			err := mockMaster.updateMasterNode()
 			Convey("No error should be returned", func() {
 				So(err, ShouldBeNil)
 			})
@@ -176,7 +184,7 @@ func TestUpdateMasterNode(t *testing.T) {
 		mockErr := errors.New("mock-error")
 		Convey("When getting API client fails", func() {
 			mockHelper.On("GetClient").Return(mockClient, mockErr)
-			err := updateMasterNode(mockHelper)
+			err := mockMaster.updateMasterNode()
 			Convey("An error should be returned", func() {
 				So(err, ShouldEqual, mockErr)
 			})
@@ -185,7 +193,7 @@ func TestUpdateMasterNode(t *testing.T) {
 		Convey("When getting API node object fails", func() {
 			mockHelper.On("GetClient").Return(mockClient, nil)
 			mockHelper.On("GetNode", mockClient, mockNodeName).Return(mockNode, mockErr)
-			err := updateMasterNode(mockHelper)
+			err := mockMaster.updateMasterNode()
 			Convey("An error should be returned", func() {
 				So(err, ShouldEqual, mockErr)
 			})
@@ -195,7 +203,7 @@ func TestUpdateMasterNode(t *testing.T) {
 			mockHelper.On("GetClient").Return(mockClient, nil)
 			mockHelper.On("GetNode", mockClient, mockNodeName).Return(mockNode, nil)
 			mockHelper.On("PatchNode", mockClient, mockNodeName, mock.Anything).Return(mockErr)
-			err := updateMasterNode(mockHelper)
+			err := mockMaster.updateMasterNode()
 			Convey("An error should be returned", func() {
 				So(err, ShouldEqual, mockErr)
 			})
@@ -205,10 +213,11 @@ func TestUpdateMasterNode(t *testing.T) {
 
 func TestAddingExtResources(t *testing.T) {
 	Convey("When adding extended resources", t, func() {
+		mockMaster := newMockMaster(nil)
 		Convey("When there are no matching labels", func() {
 			mockNode := newMockNode()
 			mockResourceLabels := ExtendedResources{}
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(len(patches), ShouldEqual, 0)
 		})
 
@@ -219,7 +228,7 @@ func TestAddingExtResources(t *testing.T) {
 				apihelper.NewJsonPatch("add", "/status/capacity", "feature-1", "1"),
 				apihelper.NewJsonPatch("add", "/status/capacity", "feature-2", "2"),
 			}
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(sortJsonPatches(patches), ShouldResemble, sortJsonPatches(expectedPatches))
 		})
 
@@ -227,7 +236,7 @@ func TestAddingExtResources(t *testing.T) {
 			mockNode := newMockNode()
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-1")] = *resource.NewQuantity(1, resource.BinarySI)
 			mockResourceLabels := ExtendedResources{LabelNs + "/feature-1": "1"}
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(len(patches), ShouldEqual, 0)
 		})
 
@@ -239,7 +248,7 @@ func TestAddingExtResources(t *testing.T) {
 				apihelper.NewJsonPatch("replace", "/status/capacity", "feature-1", "1"),
 				apihelper.NewJsonPatch("replace", "/status/allocatable", "feature-1", "1"),
 			}
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(sortJsonPatches(patches), ShouldResemble, sortJsonPatches(expectedPatches))
 		})
 	})
@@ -247,22 +256,23 @@ func TestAddingExtResources(t *testing.T) {
 
 func TestRemovingExtResources(t *testing.T) {
 	Convey("When removing extended resources", t, func() {
+		mockMaster := newMockMaster(nil)
 		Convey("When none are removed", func() {
 			mockNode := newMockNode()
 			mockResourceLabels := ExtendedResources{LabelNs + "/feature-1": "1", LabelNs + "/feature-2": "2"}
-			mockNode.Annotations[AnnotationNs+"/extended-resources"] = "feature-1,feature-2"
+			mockNode.Annotations[AnnotationNsBase+"/extended-resources"] = "feature-1,feature-2"
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-1")] = *resource.NewQuantity(1, resource.BinarySI)
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-2")] = *resource.NewQuantity(2, resource.BinarySI)
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(len(patches), ShouldEqual, 0)
 		})
 		Convey("When the related label is gone", func() {
 			mockNode := newMockNode()
 			mockResourceLabels := ExtendedResources{LabelNs + "/feature-4": "", LabelNs + "/feature-2": "2"}
-			mockNode.Annotations[AnnotationNs+"/extended-resources"] = "feature-4,feature-2"
+			mockNode.Annotations[AnnotationNsBase+"/extended-resources"] = "feature-4,feature-2"
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-4")] = *resource.NewQuantity(4, resource.BinarySI)
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-2")] = *resource.NewQuantity(2, resource.BinarySI)
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(len(patches), ShouldBeGreaterThan, 0)
 		})
 		Convey("When the extended resource is no longer wanted", func() {
@@ -270,8 +280,8 @@ func TestRemovingExtResources(t *testing.T) {
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-1")] = *resource.NewQuantity(1, resource.BinarySI)
 			mockNode.Status.Capacity[api.ResourceName(LabelNs+"/feature-2")] = *resource.NewQuantity(2, resource.BinarySI)
 			mockResourceLabels := ExtendedResources{LabelNs + "/feature-2": "2"}
-			mockNode.Annotations[AnnotationNs+"/extended-resources"] = "feature-1,feature-2"
-			patches := createExtendedResourcePatches(mockNode, mockResourceLabels)
+			mockNode.Annotations[AnnotationNsBase+"/extended-resources"] = "feature-1,feature-2"
+			patches := mockMaster.createExtendedResourcePatches(mockNode, mockResourceLabels)
 			So(len(patches), ShouldBeGreaterThan, 0)
 		})
 	})
@@ -282,9 +292,9 @@ func TestSetLabels(t *testing.T) {
 		const workerName = "mock-worker"
 		const workerVer = "0.1-test"
 		mockHelper := &apihelper.MockAPIHelpers{}
+		mockMaster := newMockMaster(mockHelper)
 		mockClient := &k8sclient.Clientset{}
 		mockNode := newMockNode()
-		mockServer := labelerServer{args: Args{LabelWhiteList: regexp.MustCompile("")}, apiHelper: mockHelper}
 		mockCtx := context.Background()
 		// In the gRPC request the label names may omit the default ns
 		mockLabels := map[string]string{"feature-1": "1", "feature-2": "val-2", "feature-3": "3"}
@@ -298,11 +308,14 @@ func TestSetLabels(t *testing.T) {
 
 		expectedStatusPatches := []apihelper.JsonPatch{}
 
+		wvAnnotation := path.Join(AnnotationNsBase, workerVersionAnnotation)
+		flAnnotation := path.Join(AnnotationNsBase, featureLabelAnnotation)
+		erAnnotation := path.Join(AnnotationNsBase, extendedResourceAnnotation)
 		Convey("When node update succeeds", func() {
 			expectedPatches := []apihelper.JsonPatch{
-				apihelper.NewJsonPatch("add", "/metadata/annotations", workerVersionAnnotation, workerVer),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", featureLabelAnnotation, strings.Join(mockLabelNames, ",")),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", extendedResourceAnnotation, ""),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", wvAnnotation, workerVer),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", flAnnotation, strings.Join(mockLabelNames, ",")),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", erAnnotation, ""),
 			}
 			for k, v := range mockLabels {
 				expectedPatches = append(expectedPatches, apihelper.NewJsonPatch("add", "/metadata/labels", LabelNs+"/"+k, v))
@@ -312,7 +325,7 @@ func TestSetLabels(t *testing.T) {
 			mockHelper.On("GetNode", mockClient, workerName).Return(mockNode, nil)
 			mockHelper.On("PatchNode", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedPatches))).Return(nil)
 			mockHelper.On("PatchNodeStatus", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedStatusPatches))).Return(nil)
-			_, err := mockServer.SetLabels(mockCtx, mockReq)
+			_, err := mockMaster.SetLabels(mockCtx, mockReq)
 			Convey("No error should be returned", func() {
 				So(err, ShouldBeNil)
 			})
@@ -320,54 +333,56 @@ func TestSetLabels(t *testing.T) {
 
 		Convey("When --label-whitelist is specified", func() {
 			expectedPatches := []apihelper.JsonPatch{
-				apihelper.NewJsonPatch("add", "/metadata/annotations", workerVersionAnnotation, workerVer),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", featureLabelAnnotation, "feature-2"),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", extendedResourceAnnotation, ""),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", wvAnnotation, workerVer),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", flAnnotation, "feature-2"),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", erAnnotation, ""),
 				apihelper.NewJsonPatch("add", "/metadata/labels", LabelNs+"/feature-2", mockLabels["feature-2"]),
 			}
 
-			mockServer.args.LabelWhiteList = regexp.MustCompile("^f.*2$")
+			mockMaster.args.LabelWhiteList = regexp.MustCompile("^f.*2$")
 			mockHelper.On("GetClient").Return(mockClient, nil)
 			mockHelper.On("GetNode", mockClient, workerName).Return(mockNode, nil)
 			mockHelper.On("PatchNode", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedPatches))).Return(nil)
 			mockHelper.On("PatchNodeStatus", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedStatusPatches))).Return(nil)
-			_, err := mockServer.SetLabels(mockCtx, mockReq)
+			_, err := mockMaster.SetLabels(mockCtx, mockReq)
 			Convey("Error is nil", func() {
 				So(err, ShouldBeNil)
 			})
 		})
 
-		Convey("When --extra-label-ns is specified", func() {
+		Convey("When --extra-label-ns and --instance are specified", func() {
 			// In the gRPC request the label names may omit the default ns
+			instance := "foo"
 			mockLabels := map[string]string{"feature-1": "val-1",
 				"valid.ns/feature-2":   "val-2",
 				"invalid.ns/feature-3": "val-3"}
 			expectedPatches := []apihelper.JsonPatch{
-				apihelper.NewJsonPatch("add", "/metadata/annotations", workerVersionAnnotation, workerVer),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", featureLabelAnnotation, "feature-1,valid.ns/feature-2"),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", extendedResourceAnnotation, ""),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", instance+"."+wvAnnotation, workerVer),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", instance+"."+flAnnotation, "feature-1,valid.ns/feature-2"),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", instance+"."+erAnnotation, ""),
 				apihelper.NewJsonPatch("add", "/metadata/labels", LabelNs+"/feature-1", mockLabels["feature-1"]),
 				apihelper.NewJsonPatch("add", "/metadata/labels", "valid.ns/feature-2", mockLabels["valid.ns/feature-2"]),
 			}
 
-			mockServer.args.ExtraLabelNs = map[string]struct{}{"valid.ns": struct{}{}}
+			mockMaster.args.ExtraLabelNs = map[string]struct{}{"valid.ns": struct{}{}}
+			mockMaster.annotationNs = instance + "." + AnnotationNsBase
 			mockHelper.On("GetClient").Return(mockClient, nil)
 			mockHelper.On("GetNode", mockClient, workerName).Return(mockNode, nil)
 			mockHelper.On("PatchNode", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedPatches))).Return(nil)
 			mockHelper.On("PatchNodeStatus", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedStatusPatches))).Return(nil)
 			mockReq := &labeler.SetLabelsRequest{NodeName: workerName, NfdVersion: workerVer, Labels: mockLabels}
-			_, err := mockServer.SetLabels(mockCtx, mockReq)
+			_, err := mockMaster.SetLabels(mockCtx, mockReq)
 			Convey("Error is nil", func() {
 				So(err, ShouldBeNil)
 			})
-
+			mockMaster.annotationNs = AnnotationNsBase
 		})
 
 		Convey("When --resource-labels is specified", func() {
 			expectedPatches := []apihelper.JsonPatch{
-				apihelper.NewJsonPatch("add", "/metadata/annotations", workerVersionAnnotation, workerVer),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", featureLabelAnnotation, "feature-2"),
-				apihelper.NewJsonPatch("add", "/metadata/annotations", extendedResourceAnnotation, "feature-1,feature-3"),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", wvAnnotation, workerVer),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", flAnnotation, "feature-2"),
+				apihelper.NewJsonPatch("add", "/metadata/annotations", erAnnotation, "feature-1,feature-3"),
 				apihelper.NewJsonPatch("add", "/metadata/labels", LabelNs+"/feature-2", mockLabels["feature-2"]),
 			}
 			expectedStatusPatches := []apihelper.JsonPatch{
@@ -375,12 +390,12 @@ func TestSetLabels(t *testing.T) {
 				apihelper.NewJsonPatch("add", "/status/capacity", LabelNs+"/feature-3", mockLabels["feature-3"]),
 			}
 
-			mockServer.args.ResourceLabels = []string{"feature-3", "feature-1"}
+			mockMaster.args.ResourceLabels = []string{"feature-3", "feature-1"}
 			mockHelper.On("GetClient").Return(mockClient, nil)
 			mockHelper.On("GetNode", mockClient, workerName).Return(mockNode, nil)
 			mockHelper.On("PatchNode", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedPatches))).Return(nil)
 			mockHelper.On("PatchNodeStatus", mockClient, mockNodeName, mock.MatchedBy(jsonPatchMatcher(expectedStatusPatches))).Return(nil)
-			_, err := mockServer.SetLabels(mockCtx, mockReq)
+			_, err := mockMaster.SetLabels(mockCtx, mockReq)
 			Convey("Error is nil", func() {
 				So(err, ShouldBeNil)
 			})
@@ -389,15 +404,15 @@ func TestSetLabels(t *testing.T) {
 		mockErr := errors.New("mock-error")
 		Convey("When node update fails", func() {
 			mockHelper.On("GetClient").Return(mockClient, mockErr)
-			_, err := mockServer.SetLabels(mockCtx, mockReq)
+			_, err := mockMaster.SetLabels(mockCtx, mockReq)
 			Convey("An error should be returned", func() {
 				So(err, ShouldEqual, mockErr)
 			})
 		})
 
-		mockServer.args.NoPublish = true
+		mockMaster.args.NoPublish = true
 		Convey("With '--no-publish'", func() {
-			_, err := mockServer.SetLabels(mockCtx, mockReq)
+			_, err := mockMaster.SetLabels(mockCtx, mockReq)
 			Convey("Operation should succeed", func() {
 				So(err, ShouldBeNil)
 			})
diff --git a/test/e2e/node_feature_discovery.go b/test/e2e/node_feature_discovery.go
index e80b1f76c..5526786d0 100644
--- a/test/e2e/node_feature_discovery.go
+++ b/test/e2e/node_feature_discovery.go
@@ -393,7 +393,7 @@ func cleanupNode(cs clientset.Interface) {
 
 			// Remove annotations
 			for key := range node.Annotations {
-				if strings.HasPrefix(key, master.AnnotationNs) {
+				if strings.HasPrefix(key, master.AnnotationNsBase) {
 					delete(node.Annotations, key)
 					update = true
 				}
@@ -573,7 +573,7 @@ var _ = framework.KubeDescribe("[NFD] Node Feature Discovery", func() {
 						gomega.Expect(node.Annotations).To(gomega.HaveKey(k))
 					}
 					for k := range node.Annotations {
-						if strings.HasPrefix(k, master.AnnotationNs) {
+						if strings.HasPrefix(k, master.AnnotationNsBase) {
 							if _, ok := nodeConf.ExpectedAnnotationValues[k]; ok {
 								continue
 							}
@@ -587,7 +587,7 @@ var _ = framework.KubeDescribe("[NFD] Node Feature Discovery", func() {
 
 					// Node running nfd-master should have master version annotation
 					if node.Name == masterPod.Spec.NodeName {
-						gomega.Expect(node.Annotations).To(gomega.HaveKey(master.AnnotationNs + "master.version"))
+						gomega.Expect(node.Annotations).To(gomega.HaveKey(master.AnnotationNsBase + "master.version"))
 					}
 				}