Repository: knative/serving
Pull request #9297 (2020-09-03 18:33:41)

Close existing WS connection when Lease change
<!--
Request Prow to automatically lint any go code in this PR:

/lint
-->

<!-- Please include the 'why' behind your changes if no issue exists -->
## Proposed Changes

* The existing connection should be closed.
* Close the connections asynchronously and wait for the processing done. This should fix the flakiness in unit test.

/assign @vagababov @mattmoor 

knative-prow-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/knative/serving/pull/9297#" title="Author self-approved">yanweiguo</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **markusthoemmes**
You can assign the PR to them by writing `/assign @markusthoemmes` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=knative%2Fserving).

<details open>
Needs approval from an approver in each of these files:

- **[pkg/autoscaler/OWNERS](https://github.com/knative/serving/blob/master/pkg/autoscaler/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["markusthoemmes"]} -->
knative-metrics-robot: The following is the coverage report on the affected files.
Say `/test pull-knative-serving-go-coverage` to re-run this coverage report

File | Old Coverage | New Coverage | Delta
---- |:------------:|:------------:|:-----:
[pkg/autoscaler/statforwarder/forwarder.go](https://storage.cloud.google.com/knative-prow/pr-logs/pull/knative_serving/9297/pull-knative-serving-go-coverage/1301589506234781698/artifacts/line-cov.html#file0) | 78.3% | 79.5% | 1.2

knative-prow-robot: @yanweiguo: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-go-coverage | 40af36454bec5226458e8ea3d70c22526b0350eb | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/9297/pull-knative-serving-go-coverage/1301589506234781698) | `/test pull-knative-serving-go-coverage`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/9297). [Your PR dashboard](https://gubernator.knative.dev/pr/yanweiguo).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
knative-prow-robot: @yanweiguo: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-go-coverage | 40af36454bec5226458e8ea3d70c22526b0350eb | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/9297/pull-knative-serving-go-coverage/1301589506234781698) | `/test pull-knative-serving-go-coverage`
pull-knative-serving-unit-tests | 40af36454bec5226458e8ea3d70c22526b0350eb | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/9297/pull-knative-serving-unit-tests/1301589506234781699) | `/test pull-knative-serving-unit-tests`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/9297). [Your PR dashboard](https://gubernator.knative.dev/pr/yanweiguo).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
knative-test-reporter-robot: <!--[AUTOMATED-FLAKY-RETRYER]40af36454bec5226458e8ea3d70c22526b0350eb[AUTOMATED-FLAKY-RETRYER]-->
The following jobs failed:

Test name | Triggers | Retries
--- | --- | ---
pull-knative-serving-unit-tests |  | 0/3

Failed non-flaky tests preventing automatic retry of pull-knative-serving-unit-tests:

```
_go_tests.DataRaceAnalysis
pkg/autoscaler/statforwarder.TestForwarderReconcile
```

Patch:
diff --git a/pkg/activator/net/throttler.go b/pkg/activator/net/throttler.go
index 5acb46842..368b96b56 100644
--- a/pkg/activator/net/throttler.go
+++ b/pkg/activator/net/throttler.go
@@ -275,8 +275,12 @@ func (rt *revisionThrottler) updateCapacity(backendCount int) {
 			return 0
 		}
 
+		// Sort, so we get more or less stable results.
+		sort.Slice(rt.podTrackers, func(i, j int) bool {
+			return rt.podTrackers[i].dest < rt.podTrackers[j].dest
+		})
 		assigned := rt.podTrackers
-		if rt.containerConcurrency != 0 {
+		if rt.containerConcurrency > 0 {
 			rt.resetTrackers()
 			assigned = assignSlice(rt.podTrackers, ai, ac, rt.containerConcurrency)
 		}
@@ -334,8 +338,8 @@ func (rt *revisionThrottler) updateThrottlerState(
 // pickIndices picks the indices for the slicing.
 func pickIndices(numTrackers, selfIndex, numActivators int) (beginIndex, endIndex, remnants int) {
 	if numActivators > numTrackers {
-		// 1. We have fewer pods than than activators. Assign the pod in round robin fashion.
-		// NB: when we implement subsetting this will be less of a problem.
+		// 1. We have fewer pods than than activators. Assign the pods in round robin fashion.
+		// With subsetting this is less of a problem and should almost never happen.
 		// e.g. lt=3, #ac = 5; for selfIdx = 3 => 3 % 3 = 0, or for si = 5 => 5%3 = 2
 		beginIndex = selfIndex % numTrackers
 		endIndex = beginIndex + 1
@@ -354,16 +358,19 @@ func pickIndices(numTrackers, selfIndex, numActivators int) (beginIndex, endInde
 // assignSlice picks a subset of the individual pods to send requests to
 // for this Activator instance. This only matters in case of direct
 // to pod IP routing, and is irrelevant, when ClusterIP is used.
+// assignSlice should receive podTrackers sorted by address.
 func assignSlice(trackers []*podTracker, selfIndex, numActivators, cc int) []*podTracker {
 	// When we're unassigned, doesn't matter what we return.
 	lt := len(trackers)
 	if selfIndex == -1 || lt <= 1 {
 		return trackers
 	}
-	// Sort, so we get more or less stable results.
-	sort.Slice(trackers, func(i, j int) bool {
-		return trackers[i].dest < trackers[j].dest
-	})
+
+	// If there's just a single activator. Take all the trackers.
+	if numActivators == 1 {
+		return trackers
+	}
+
 	bi, ei, remnants := pickIndices(lt, selfIndex, numActivators)
 	x := append(trackers[:0:0], trackers[bi:ei]...)
 	if remnants > 0 {
@@ -374,8 +381,8 @@ func assignSlice(trackers []*podTracker, selfIndex, numActivators, cc int) []*po
 		rand.Shuffle(remnants, func(i, j int) {
 			tail[i], tail[j] = tail[j], tail[i]
 		})
-		// We need minOneOrValue in order for cc==0 to work.
-		dcc := minOneOrValue(int(math.Ceil(float64(cc) / float64(numActivators))))
+		// assignSlice is not called for CC=0 case.
+		dcc := int(math.Ceil(float64(cc) / float64(numActivators)))
 		// This is basically: x = append(x, trackers[len(trackers)-remnants:]...)
 		// But we need to update the capacity.
 		for _, t := range tail {
@@ -386,7 +393,7 @@ func assignSlice(trackers []*podTracker, selfIndex, numActivators, cc int) []*po
 	return x
 }
 
-// This function will never be called in parallel but try can be called in parallel to this so we need
+// This function will never be called in parallel but `try` can be called in parallel to this so we need
 // to lock on updating concurrency / trackers
 func (rt *revisionThrottler) handleUpdate(throttler *Throttler, update revisionDestsUpdate) {
 	rt.logger.Debugw("Handling update",
diff --git a/pkg/apis/serving/metadata_validation.go b/pkg/apis/serving/metadata_validation.go
index 0af0e595a..42685dda7 100644
--- a/pkg/apis/serving/metadata_validation.go
+++ b/pkg/apis/serving/metadata_validation.go
@@ -115,9 +115,9 @@ func ValidateContainerConcurrency(ctx context.Context, containerConcurrency *int
 }
 
 // ValidateClusterVisibilityLabel function validates the visibility label on a Route
-func ValidateClusterVisibilityLabel(label string) (errs *apis.FieldError) {
+func ValidateClusterVisibilityLabel(label, key string) (errs *apis.FieldError) {
 	if label != VisibilityClusterLocal {
-		errs = apis.ErrInvalidValue(label, VisibilityLabelKey)
+		errs = apis.ErrInvalidValue(label, key)
 	}
 	return
 }
diff --git a/pkg/apis/serving/register.go b/pkg/apis/serving/register.go
index 94eb21cef..c140bf452 100644
--- a/pkg/apis/serving/register.go
+++ b/pkg/apis/serving/register.go
@@ -90,11 +90,10 @@ const (
 	// It has to be in [0.1,100]
 	QueueSideCarResourcePercentageAnnotation = "queue.sidecar." + GroupName + "/resourcePercentage"
 
-	// VisibilityLabelKey is the label to indicate visibility of Route
-	// and KServices.  It can be an annotation too but since users are
-	// already using labels for domain, it probably best to keep this
-	// consistent.
-	VisibilityLabelKey = "serving.knative.dev/visibility"
+	// VisibilityLabelKeyObsolete is the obsolete VisibilityLabelKey.
+	// This will move over to VisibilityLabelKey in networking repo..
+	VisibilityLabelKeyObsolete = "serving.knative.dev/visibility"
+
 	// VisibilityClusterLocal is the label value for VisibilityLabelKey
 	// that will result to the Route/KService getting a cluster local
 	// domain suffix.
diff --git a/pkg/apis/serving/v1/configuration_validation.go b/pkg/apis/serving/v1/configuration_validation.go
index cc5a3811f..f2ea6f752 100644
--- a/pkg/apis/serving/v1/configuration_validation.go
+++ b/pkg/apis/serving/v1/configuration_validation.go
@@ -74,10 +74,8 @@ func (csf *ConfigurationStatusFields) Validate(ctx context.Context) *apis.FieldE
 func (c *Configuration) validateLabels() (errs *apis.FieldError) {
 	for key, val := range c.GetLabels() {
 		switch key {
-		case serving.RouteLabelKey:
+		case serving.RouteLabelKey, serving.VisibilityLabelKeyObsolete:
 			// Known valid labels.
-		case serving.VisibilityLabelKey:
-			errs = errs.Also(validateClusterVisibilityLabel(val))
 		case serving.ServiceLabelKey:
 			errs = errs.Also(verifyLabelOwnerRef(val, serving.ServiceLabelKey, "Service", c.GetOwnerReferences()))
 		default:
diff --git a/pkg/apis/serving/v1/route_validation.go b/pkg/apis/serving/v1/route_validation.go
index e9d2ba2f9..936d5f411 100644
--- a/pkg/apis/serving/v1/route_validation.go
+++ b/pkg/apis/serving/v1/route_validation.go
@@ -22,6 +22,7 @@ import (
 	"strings"
 
 	"k8s.io/apimachinery/pkg/util/validation"
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/serving/pkg/apis/serving"
 )
@@ -208,7 +209,7 @@ func (rsf *RouteStatusFields) Validate(ctx context.Context) *apis.FieldError {
 
 func validateClusterVisibilityLabel(label string) (errs *apis.FieldError) {
 	if label != serving.VisibilityClusterLocal {
-		errs = apis.ErrInvalidValue(label, serving.VisibilityLabelKey)
+		errs = apis.ErrInvalidValue(label, network.VisibilityLabelKey)
 	}
 	return
 }
@@ -217,7 +218,7 @@ func validateClusterVisibilityLabel(label string) (errs *apis.FieldError) {
 func (r *Route) validateLabels() (errs *apis.FieldError) {
 	for key, val := range r.GetLabels() {
 		switch key {
-		case serving.VisibilityLabelKey:
+		case serving.VisibilityLabelKeyObsolete, network.VisibilityLabelKey:
 			errs = errs.Also(validateClusterVisibilityLabel(val))
 		case serving.ServiceLabelKey:
 			errs = errs.Also(verifyLabelOwnerRef(val, serving.ServiceLabelKey, "Service", r.GetOwnerReferences()))
diff --git a/pkg/apis/serving/v1/service_validation.go b/pkg/apis/serving/v1/service_validation.go
index f6c4ddc8e..a95ebf691 100644
--- a/pkg/apis/serving/v1/service_validation.go
+++ b/pkg/apis/serving/v1/service_validation.go
@@ -20,6 +20,7 @@ import (
 	"context"
 	"strings"
 
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/serving/pkg/apis/serving"
 )
@@ -68,7 +69,7 @@ func (ss *ServiceStatus) Validate(ctx context.Context) *apis.FieldError {
 func (s *Service) validateLabels() (errs *apis.FieldError) {
 	for key, val := range s.GetLabels() {
 		switch {
-		case key == serving.VisibilityLabelKey:
+		case key == network.VisibilityLabelKey || key == serving.VisibilityLabelKeyObsolete:
 			errs = errs.Also(validateClusterVisibilityLabel(val))
 		case strings.HasPrefix(key, serving.GroupNamePrefix):
 			errs = errs.Also(apis.ErrInvalidKeyName(key, apis.CurrentField))
diff --git a/pkg/apis/serving/v1beta1/configuration_validation.go b/pkg/apis/serving/v1beta1/configuration_validation.go
index fbc2a9666..8f059c369 100644
--- a/pkg/apis/serving/v1beta1/configuration_validation.go
+++ b/pkg/apis/serving/v1beta1/configuration_validation.go
@@ -59,10 +59,8 @@ func (c *Configuration) Validate(ctx context.Context) (errs *apis.FieldError) {
 func (c *Configuration) validateLabels() (errs *apis.FieldError) {
 	for key, val := range c.GetLabels() {
 		switch key {
-		case serving.RouteLabelKey:
+		case serving.RouteLabelKey, serving.VisibilityLabelKeyObsolete:
 			// Known valid labels.
-		case serving.VisibilityLabelKey:
-			errs = errs.Also(serving.ValidateClusterVisibilityLabel(val))
 		case serving.ServiceLabelKey:
 			errs = errs.Also(verifyLabelOwnerRef(val, serving.ServiceLabelKey, "Service", c.GetOwnerReferences()))
 		default:
diff --git a/pkg/apis/serving/v1beta1/route_validation.go b/pkg/apis/serving/v1beta1/route_validation.go
index cd2832170..498d47f95 100644
--- a/pkg/apis/serving/v1beta1/route_validation.go
+++ b/pkg/apis/serving/v1beta1/route_validation.go
@@ -20,6 +20,7 @@ import (
 	"context"
 	"strings"
 
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/serving/pkg/apis/serving"
 )
@@ -47,8 +48,8 @@ func (r *Route) Validate(ctx context.Context) *apis.FieldError {
 func (r *Route) validateLabels() (errs *apis.FieldError) {
 	for key, val := range r.GetLabels() {
 		switch key {
-		case serving.VisibilityLabelKey:
-			errs = errs.Also(serving.ValidateClusterVisibilityLabel(val))
+		case network.VisibilityLabelKey, serving.VisibilityLabelKeyObsolete:
+			errs = errs.Also(serving.ValidateClusterVisibilityLabel(val, key))
 		case serving.ServiceLabelKey:
 			errs = errs.Also(verifyLabelOwnerRef(val, serving.ServiceLabelKey, "Service", r.GetOwnerReferences()))
 		default:
diff --git a/pkg/apis/serving/v1beta1/service_validation.go b/pkg/apis/serving/v1beta1/service_validation.go
index 9204d5745..96766402d 100644
--- a/pkg/apis/serving/v1beta1/service_validation.go
+++ b/pkg/apis/serving/v1beta1/service_validation.go
@@ -20,6 +20,7 @@ import (
 	"context"
 	"strings"
 
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/serving/pkg/apis/serving"
 )
@@ -54,8 +55,8 @@ func (s *Service) Validate(ctx context.Context) (errs *apis.FieldError) {
 func (s *Service) validateLabels() (errs *apis.FieldError) {
 	for key, val := range s.GetLabels() {
 		switch {
-		case key == serving.VisibilityLabelKey:
-			errs = errs.Also(serving.ValidateClusterVisibilityLabel(val))
+		case key == network.VisibilityLabelKey || key == serving.VisibilityLabelKeyObsolete:
+			errs = errs.Also(serving.ValidateClusterVisibilityLabel(val, key))
 		case strings.HasPrefix(key, serving.GroupNamePrefix):
 			errs = errs.Also(apis.ErrInvalidKeyName(key, apis.CurrentField))
 		}
diff --git a/pkg/autoscaler/statforwarder/forwarder.go b/pkg/autoscaler/statforwarder/forwarder.go
index a05837f4c..6b168a79b 100644
--- a/pkg/autoscaler/statforwarder/forwarder.go
+++ b/pkg/autoscaler/statforwarder/forwarder.go
@@ -24,7 +24,6 @@ import (
 
 	gorillawebsocket "github.com/gorilla/websocket"
 	"go.uber.org/zap"
-
 	coordinationv1 "k8s.io/api/coordination/v1"
 	v1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/api/equality"
@@ -35,7 +34,6 @@ import (
 	"k8s.io/client-go/kubernetes"
 	corev1listers "k8s.io/client-go/listers/core/v1"
 	"k8s.io/client-go/tools/cache"
-
 	leaseinformer "knative.dev/pkg/client/injection/kube/informers/coordination/v1/lease"
 	endpointsinformer "knative.dev/pkg/client/injection/kube/informers/core/v1/endpoints"
 	"knative.dev/pkg/controller"
@@ -86,6 +84,9 @@ type Forwarder struct {
 	// processorsLock is the lock for processors.
 	processorsLock sync.RWMutex
 	processors     map[string]*bucketProcessor
+	// Used to capture asynchronous processes to be waited
+	// on when shutting the WebSocket connection down.
+	processingWg sync.WaitGroup
 }
 
 // New creates a new Forwarder.
@@ -154,6 +155,9 @@ func (f *Forwarder) leaseUpdated(obj interface{}) {
 		return
 	}
 
+	// Close existing connection if there's one. The target address won't
+	// be the same as the IP has changed.
+	f.shutdown(f.getProcessor(n))
 	holder := *l.Spec.HolderIdentity
 	f.setProcessor(n, f.createProcessor(n, holder))
 
@@ -305,6 +309,7 @@ func (f *Forwarder) createProcessor(bkt, holder string) *bucketProcessor {
 	dns := fmt.Sprintf("ws://%s:%d", holder, autoscalerPort)
 	f.logger.Info("Connecting to Autoscaler bucket at ", dns)
 	c := websocket.NewDurableSendingConnection(dns, f.logger)
+	f.processingWg.Add(1)
 	return &bucketProcessor{
 		holder: holder,
 		conn:   c,
@@ -342,11 +347,22 @@ func (f *Forwarder) Process(sm asmetrics.StatMessage) {
 	p.proc(sm)
 }
 
+func (f *Forwarder) shutdown(p *bucketProcessor) {
+	if p != nil && p.conn != nil {
+		go func() {
+			defer f.processingWg.Done()
+			p.conn.Shutdown()
+		}()
+	}
+}
+
 // Cancel is the function to call when terminating a Forwarder.
 func (f *Forwarder) Cancel() {
+	f.processorsLock.RLock()
+	defer f.processorsLock.RUnlock()
+
 	for _, p := range f.processors {
-		if p.conn != nil {
-			p.conn.Shutdown()
-		}
+		f.shutdown(p)
 	}
+	f.processingWg.Wait()
 }
diff --git a/pkg/reconciler/route/config/domain.go b/pkg/reconciler/route/config/domain.go
index cc88260a5..674bc0d87 100644
--- a/pkg/reconciler/route/config/domain.go
+++ b/pkg/reconciler/route/config/domain.go
@@ -22,6 +22,7 @@ import (
 	"github.com/ghodss/yaml"
 	corev1 "k8s.io/api/core/v1"
 
+	netpkg "knative.dev/networking/pkg"
 	"knative.dev/pkg/configmap"
 	"knative.dev/pkg/network"
 	"knative.dev/serving/pkg/apis/serving"
@@ -98,7 +99,8 @@ func (c *Domain) LookupDomainForLabels(labels map[string]string) string {
 	specificity := -1
 	// If we see VisibilityLabelKey sets with VisibilityClusterLocal, that
 	// will take precedence and the route will get a Cluster's Domain Name.
-	if labels[serving.VisibilityLabelKey] == serving.VisibilityClusterLocal {
+	if labels[netpkg.VisibilityLabelKey] == serving.VisibilityClusterLocal ||
+		labels[serving.VisibilityLabelKeyObsolete] == serving.VisibilityClusterLocal {
 		return "svc." + network.GetClusterDomainName()
 	}
 	for k, selector := range c.Domains {
diff --git a/pkg/reconciler/route/domains/domains.go b/pkg/reconciler/route/domains/domains.go
index d4c122863..a1cc7fa10 100644
--- a/pkg/reconciler/route/domains/domains.go
+++ b/pkg/reconciler/route/domains/domains.go
@@ -84,7 +84,8 @@ func DomainNameFromTemplate(ctx context.Context, r metav1.ObjectMeta, name strin
 	var templ *template.Template
 	// If the route is "cluster local" then don't use the user-defined
 	// domain template, use the default one
-	if rLabels[serving.VisibilityLabelKey] == serving.VisibilityClusterLocal {
+	if rLabels[network.VisibilityLabelKey] == serving.VisibilityClusterLocal ||
+		rLabels[serving.VisibilityLabelKeyObsolete] == serving.VisibilityClusterLocal {
 		templ = template.Must(template.New("domain-template").Parse(
 			network.DefaultDomainTemplate))
 	} else {
diff --git a/pkg/reconciler/route/resources/filters.go b/pkg/reconciler/route/resources/filters.go
index 6b7225a31..7c1f9809d 100644
--- a/pkg/reconciler/route/resources/filters.go
+++ b/pkg/reconciler/route/resources/filters.go
@@ -18,10 +18,12 @@ package resources
 
 import (
 	corev1 "k8s.io/api/core/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/serving/pkg/apis/serving"
 )
 
 // IsClusterLocalService returns whether a service is cluster local.
 func IsClusterLocalService(svc *corev1.Service) bool {
-	return svc.GetLabels()[serving.VisibilityLabelKey] == serving.VisibilityClusterLocal
+	return svc.GetLabels()[network.VisibilityLabelKey] == serving.VisibilityClusterLocal ||
+		svc.GetLabels()[serving.VisibilityLabelKeyObsolete] == serving.VisibilityClusterLocal
 }
diff --git a/pkg/reconciler/route/resources/labels/labels.go b/pkg/reconciler/route/resources/labels/labels.go
index c6cbeee38..865360f4e 100644
--- a/pkg/reconciler/route/resources/labels/labels.go
+++ b/pkg/reconciler/route/resources/labels/labels.go
@@ -18,20 +18,21 @@ package labels
 
 import (
 	v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/serving/pkg/apis/serving"
 )
 
 // IsObjectLocalVisibility returns whether an ObjectMeta is of cluster-local visibility
 func IsObjectLocalVisibility(meta *v1.ObjectMeta) bool {
-	return meta.Labels[serving.VisibilityLabelKey] != ""
+	return meta.Labels[network.VisibilityLabelKey] != "" || meta.Labels[serving.VisibilityLabelKeyObsolete] != ""
 }
 
 // SetVisibility sets the visibility on an ObjectMeta
 func SetVisibility(meta *v1.ObjectMeta, isClusterLocal bool) {
 	if isClusterLocal {
-		SetLabel(meta, serving.VisibilityLabelKey, serving.VisibilityClusterLocal)
+		SetLabel(meta, network.VisibilityLabelKey, serving.VisibilityClusterLocal)
 	} else {
-		DeleteLabel(meta, serving.VisibilityLabelKey)
+		DeleteLabel(meta, network.VisibilityLabelKey)
 	}
 }
 
diff --git a/pkg/reconciler/route/resources/service.go b/pkg/reconciler/route/resources/service.go
index 90c338751..d9096f56b 100644
--- a/pkg/reconciler/route/resources/service.go
+++ b/pkg/reconciler/route/resources/service.go
@@ -26,6 +26,7 @@ import (
 	"k8s.io/apimachinery/pkg/util/intstr"
 	"k8s.io/apimachinery/pkg/util/sets"
 
+	network "knative.dev/networking/pkg"
 	"knative.dev/networking/pkg/apis/networking"
 	netv1alpha1 "knative.dev/networking/pkg/apis/networking/v1alpha1"
 	"knative.dev/pkg/kmeta"
@@ -120,7 +121,7 @@ func makeK8sService(ctx context.Context, route *v1.Route, targetName string) (*c
 			Labels: kmeta.UnionMaps(kmeta.FilterMap(route.GetLabels(), func(key string) bool {
 				// Do not propagate the visibility label from Route as users may want to set the label
 				// in the specific k8s svc for subroute. see https://github.com/knative/serving/pull/4560.
-				return key == serving.VisibilityLabelKey
+				return (key == network.VisibilityLabelKey || key == serving.VisibilityLabelKeyObsolete)
 			}), svcLabels),
 			Annotations: route.GetAnnotations(),
 		},


Test patch:
diff --git a/pkg/activator/net/throttler_test.go b/pkg/activator/net/throttler_test.go
index 00557e31d..357b640f0 100644
--- a/pkg/activator/net/throttler_test.go
+++ b/pkg/activator/net/throttler_test.go
@@ -309,6 +309,15 @@ func TestThrottlerErrorOneTimesOut(t *testing.T) {
 	}
 }
 
+func sortedTrackers(trk []*podTracker) bool {
+	for i := 1; i < len(trk); i++ {
+		if trk[i].dest < trk[i-1].dest {
+			return false
+		}
+	}
+	return true
+}
+
 func TestThrottlerSuccesses(t *testing.T) {
 	for _, tc := range []struct {
 		name        string
@@ -469,13 +478,15 @@ func TestThrottlerSuccesses(t *testing.T) {
 				wantCapacity = dests * int(*cc)
 			}
 			if err := wait.PollImmediate(10*time.Millisecond, 3*time.Second, func() (bool, error) {
+				rt.mux.RLock()
+				defer rt.mux.RUnlock()
 				if *cc != 0 {
-					return rt.activatorIndex.Load() != -1 && rt.breaker.Capacity() == wantCapacity, nil
+					return rt.activatorIndex.Load() != -1 && rt.breaker.Capacity() == wantCapacity &&
+						sortedTrackers(rt.assignedTrackers), nil
 				}
 				// If CC=0 then verify number of backends, rather the capacity of breaker.
-				rt.mux.RLock()
-				defer rt.mux.RUnlock()
-				return rt.activatorIndex.Load() != -1 && dests == len(rt.assignedTrackers), nil
+				return rt.activatorIndex.Load() != -1 && dests == len(rt.assignedTrackers) &&
+					sortedTrackers(rt.assignedTrackers), nil
 			}); err != nil {
 				t.Fatal("Timed out waiting for the capacity to be updated")
 			}
@@ -1060,22 +1071,30 @@ func TestAssignSlice(t *testing.T) {
 	opt := cmp.Comparer(func(a, b *podTracker) bool {
 		return a.dest == b.dest
 	})
+	// assignSlice receives the pod trackers sorted.
 	trackers := []*podTracker{{
-		dest: "2",
-	}, {
 		dest: "1",
+	}, {
+		dest: "2",
 	}, {
 		dest: "3",
 	}}
 	t.Run("notrackers", func(t *testing.T) {
-		got := assignSlice([]*podTracker{}, 0 /*selfIdx*/, 1 /*numAct*/, 0 /*cc*/)
+		got := assignSlice([]*podTracker{}, 0 /*selfIdx*/, 1 /*numAct*/, 42 /*cc*/)
 		if !cmp.Equal(got, []*podTracker{}, opt) {
 			t.Errorf("Got=%v, want: %v, diff: %s", got, trackers,
 				cmp.Diff([]*podTracker{}, got, opt))
 		}
 	})
+	t.Run("idx=1, na=1", func(t *testing.T) {
+		got := assignSlice(trackers, 1, 1, 1982)
+		if !cmp.Equal(got, trackers, opt) {
+			t.Errorf("Got=%v, want: %v, diff: %s", got, trackers,
+				cmp.Diff(trackers, got, opt))
+		}
+	})
 	t.Run("idx=-1", func(t *testing.T) {
-		got := assignSlice(trackers, -1, 1, 0)
+		got := assignSlice(trackers, -1, 1, 1982)
 		if !cmp.Equal(got, trackers, opt) {
 			t.Errorf("Got=%v, want: %v, diff: %s", got, trackers,
 				cmp.Diff(trackers, got, opt))
@@ -1083,14 +1102,14 @@ func TestAssignSlice(t *testing.T) {
 	})
 	t.Run("idx=1", func(t *testing.T) {
 		cp := append(trackers[:0:0], trackers...)
-		got := assignSlice(cp, 1, 3, 0)
-		if !cmp.Equal(got, trackers[0:1], opt) {
+		got := assignSlice(cp, 1, 3, 1984)
+		if !cmp.Equal(got, trackers[1:2], opt) {
 			t.Errorf("Got=%v, want: %v; diff: %s", got, trackers[0:1],
-				cmp.Diff(trackers[0:1], got, opt))
+				cmp.Diff(trackers[1:2], got, opt))
 		}
 	})
 	t.Run("len=1", func(t *testing.T) {
-		got := assignSlice(trackers[0:1], 1, 3, 0)
+		got := assignSlice(trackers[0:1], 1, 3, 1988)
 		if !cmp.Equal(got, trackers[0:1], opt) {
 			t.Errorf("Got=%v, want: %v; diff: %s", got, trackers[0:1],
 				cmp.Diff(trackers[0:1], got, opt))
@@ -1099,10 +1118,10 @@ func TestAssignSlice(t *testing.T) {
 
 	t.Run("idx=1, cc=5", func(t *testing.T) {
 		trackers := []*podTracker{{
-			dest: "2",
+			dest: "1",
 			b:    queue.NewBreaker(testBreakerParams),
 		}, {
-			dest: "1",
+			dest: "2",
 			b:    queue.NewBreaker(testBreakerParams),
 		}, {
 			dest: "3",
@@ -1110,10 +1129,10 @@ func TestAssignSlice(t *testing.T) {
 		}}
 		cp := append(trackers[:0:0], trackers...)
 		got := assignSlice(cp, 1, 2, 5)
-		want := append(trackers[0:1], trackers[2:]...)
+		want := trackers[1:3]
 		if !cmp.Equal(got, want, opt) {
 			t.Errorf("Got=%v, want: %v; diff: %s", got, want,
-				cmp.Diff(trackers[0:1], got, opt))
+				cmp.Diff(want, got, opt))
 		}
 		if got, want := got[1].b.Capacity(), 5/2+1; got != want {
 			t.Errorf("Capacity for the tail pod = %d, want: %d", got, want)
@@ -1121,10 +1140,10 @@ func TestAssignSlice(t *testing.T) {
 	})
 	t.Run("idx=1, cc=6", func(t *testing.T) {
 		trackers := []*podTracker{{
-			dest: "2",
+			dest: "1",
 			b:    queue.NewBreaker(testBreakerParams),
 		}, {
-			dest: "1",
+			dest: "2",
 			b:    queue.NewBreaker(testBreakerParams),
 		}, {
 			dest: "3",
@@ -1132,10 +1151,10 @@ func TestAssignSlice(t *testing.T) {
 		}}
 		cp := append(trackers[:0:0], trackers...)
 		got := assignSlice(cp, 1, 2, 6)
-		want := append(trackers[0:1], trackers[2:]...)
+		want := trackers[1:]
 		if !cmp.Equal(got, want, opt) {
 			t.Errorf("Got=%v, want: %v; diff: %s", got, want,
-				cmp.Diff(trackers[0:1], got, opt))
+				cmp.Diff(want, got, opt))
 		}
 		if got, want := got[1].b.Capacity(), 3; got != want {
 			t.Errorf("Capacity for the tail pod = %d, want: %d", got, want)
diff --git a/pkg/apis/serving/metadata_validation_test.go b/pkg/apis/serving/metadata_validation_test.go
index 84d1e122b..8c621f7ba 100644
--- a/pkg/apis/serving/metadata_validation_test.go
+++ b/pkg/apis/serving/metadata_validation_test.go
@@ -26,6 +26,7 @@ import (
 	authv1 "k8s.io/api/authentication/v1"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/pkg/ptr"
 	"knative.dev/serving/pkg/apis/autoscaling"
@@ -370,19 +371,19 @@ func TestValidateClusterVisibilityLabel(t *testing.T) {
 	}{{
 		name:      "empty label",
 		label:     "",
-		expectErr: apis.ErrInvalidValue("", VisibilityLabelKey),
+		expectErr: apis.ErrInvalidValue("", network.VisibilityLabelKey),
 	}, {
 		name:  "valid label",
 		label: VisibilityClusterLocal,
 	}, {
 		name:      "invalid label",
 		label:     "not-cluster-local",
-		expectErr: apis.ErrInvalidValue("not-cluster-local", VisibilityLabelKey),
+		expectErr: apis.ErrInvalidValue("not-cluster-local", network.VisibilityLabelKey),
 	}}
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			err := ValidateClusterVisibilityLabel(test.label)
+			err := ValidateClusterVisibilityLabel(test.label, network.VisibilityLabelKey)
 			if got, want := err.Error(), test.expectErr.Error(); got != want {
 				t.Errorf("\nGot:  %q\nwant: %q", got, want)
 			}
diff --git a/pkg/apis/serving/v1/configuration_validation_test.go b/pkg/apis/serving/v1/configuration_validation_test.go
index 97e70633d..99f2b81e0 100644
--- a/pkg/apis/serving/v1/configuration_validation_test.go
+++ b/pkg/apis/serving/v1/configuration_validation_test.go
@@ -268,29 +268,17 @@ func TestConfigurationLabelValidation(t *testing.T) {
 		c    *Configuration
 		want *apis.FieldError
 	}{{
-		name: "valid visibility name",
+		name: "valid obsolete visibility name",
 		c: &Configuration{
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "byo-name",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "cluster-local",
+					serving.VisibilityLabelKeyObsolete: "cluster-local",
 				},
 			},
 			Spec: validConfigSpec,
 		},
 		want: nil,
-	}, {
-		name: "invalid visibility name",
-		c: &Configuration{
-			ObjectMeta: metav1.ObjectMeta{
-				Name: "byo-name",
-				Labels: map[string]string{
-					serving.VisibilityLabelKey: "bad-value",
-				},
-			},
-			Spec: validConfigSpec,
-		},
-		want: apis.ErrInvalidValue("bad-value", "metadata.labels.serving.knative.dev/visibility"),
 	}, {
 		name: "valid route name",
 		c: &Configuration{
diff --git a/pkg/apis/serving/v1/route_validation_test.go b/pkg/apis/serving/v1/route_validation_test.go
index 2ec7af16a..2d428e7f3 100644
--- a/pkg/apis/serving/v1/route_validation_test.go
+++ b/pkg/apis/serving/v1/route_validation_test.go
@@ -23,6 +23,7 @@ import (
 
 	"github.com/google/go-cmp/cmp"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/pkg/ptr"
 	"knative.dev/serving/pkg/apis/serving"
@@ -539,7 +540,7 @@ func TestRouteLabelValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "byo-name",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "cluster-local",
+					network.VisibilityLabelKey: "cluster-local",
 				},
 			},
 			Spec: validRouteSpec,
@@ -551,12 +552,12 @@ func TestRouteLabelValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "byo-name",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "bad-value",
+					network.VisibilityLabelKey: "bad-value",
 				},
 			},
 			Spec: validRouteSpec,
 		},
-		want: apis.ErrInvalidValue("bad-value", "metadata.labels.serving.knative.dev/visibility"),
+		want: apis.ErrInvalidValue("bad-value", "metadata.labels."+network.VisibilityLabelKey),
 	}, {
 		name: "valid knative service name",
 		r: &Route{
diff --git a/pkg/apis/serving/v1/service_validation_test.go b/pkg/apis/serving/v1/service_validation_test.go
index d5ca6de2d..c0690a96a 100644
--- a/pkg/apis/serving/v1/service_validation_test.go
+++ b/pkg/apis/serving/v1/service_validation_test.go
@@ -23,6 +23,7 @@ import (
 	"github.com/google/go-cmp/cmp"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/pkg/ptr"
 	"knative.dev/serving/pkg/apis/config"
@@ -75,7 +76,7 @@ func TestServiceValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "valid",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "cluster-local",
+					network.VisibilityLabelKey: "cluster-local",
 				},
 			},
 			Spec: ServiceSpec{
@@ -120,7 +121,7 @@ func TestServiceValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "valid",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "bad-label",
+					network.VisibilityLabelKey: "bad-label",
 				},
 			},
 			Spec: ServiceSpec{
@@ -128,7 +129,7 @@ func TestServiceValidation(t *testing.T) {
 				RouteSpec:         goodRouteSpec,
 			},
 		},
-		want: apis.ErrInvalidValue("bad-label", "metadata.labels.serving.knative.dev/visibility"),
+		want: apis.ErrInvalidValue("bad-label", "metadata.labels."+network.VisibilityLabelKey),
 	}, {
 		name: "valid release",
 		r: &Service{
diff --git a/pkg/apis/serving/v1beta1/configuration_validation_test.go b/pkg/apis/serving/v1beta1/configuration_validation_test.go
index b72c477ac..394e37d8f 100644
--- a/pkg/apis/serving/v1beta1/configuration_validation_test.go
+++ b/pkg/apis/serving/v1beta1/configuration_validation_test.go
@@ -246,29 +246,18 @@ func TestConfigurationLabelValidation(t *testing.T) {
 		c    *Configuration
 		want *apis.FieldError
 	}{{
-		name: "valid visibility name",
+
+		name: "valid obsolete visibility name",
 		c: &Configuration{
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "byo-name",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "cluster-local",
+					serving.VisibilityLabelKeyObsolete: "cluster-local",
 				},
 			},
 			Spec: validConfigSpec,
 		},
 		want: nil,
-	}, {
-		name: "invalid visibility name",
-		c: &Configuration{
-			ObjectMeta: metav1.ObjectMeta{
-				Name: "byo-name",
-				Labels: map[string]string{
-					serving.VisibilityLabelKey: "bad-value",
-				},
-			},
-			Spec: validConfigSpec,
-		},
-		want: apis.ErrInvalidValue("bad-value", "metadata.labels.serving.knative.dev/visibility"),
 	}, {
 		name: "valid route name",
 		c: &Configuration{
diff --git a/pkg/apis/serving/v1beta1/route_validation_test.go b/pkg/apis/serving/v1beta1/route_validation_test.go
index 149ec3c8c..ed693c293 100644
--- a/pkg/apis/serving/v1beta1/route_validation_test.go
+++ b/pkg/apis/serving/v1beta1/route_validation_test.go
@@ -24,6 +24,7 @@ import (
 	"github.com/google/go-cmp/cmp"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis"
 	"knative.dev/pkg/ptr"
 	"knative.dev/serving/pkg/apis/serving"
@@ -507,7 +508,7 @@ func TestRouteLabelValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "byo-name",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "cluster-local",
+					network.VisibilityLabelKey: "cluster-local",
 				},
 			},
 			Spec: validRouteSpec,
@@ -519,12 +520,12 @@ func TestRouteLabelValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "byo-name",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "bad-value",
+					network.VisibilityLabelKey: "bad-value",
 				},
 			},
 			Spec: validRouteSpec,
 		},
-		want: apis.ErrInvalidValue("bad-value", "metadata.labels.serving.knative.dev/visibility"),
+		want: apis.ErrInvalidValue("bad-value", "metadata.labels."+network.VisibilityLabelKey),
 	}, {
 		name: "valid knative service name",
 		r: &Route{
diff --git a/pkg/apis/serving/v1beta1/service_validation_test.go b/pkg/apis/serving/v1beta1/service_validation_test.go
index caeffede5..5a4618b09 100644
--- a/pkg/apis/serving/v1beta1/service_validation_test.go
+++ b/pkg/apis/serving/v1beta1/service_validation_test.go
@@ -23,6 +23,7 @@ import (
 	"github.com/google/go-cmp/cmp"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/pkg/ptr"
 	"knative.dev/serving/pkg/apis/config"
 	"knative.dev/serving/pkg/apis/serving"
@@ -77,7 +78,7 @@ func TestServiceValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "valid",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "cluster-local",
+					network.VisibilityLabelKey: "cluster-local",
 				},
 			},
 			Spec: v1.ServiceSpec{
@@ -122,7 +123,7 @@ func TestServiceValidation(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "valid",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "bad-label",
+					network.VisibilityLabelKey: "bad-label",
 				},
 			},
 			Spec: v1.ServiceSpec{
@@ -130,7 +131,7 @@ func TestServiceValidation(t *testing.T) {
 				RouteSpec:         goodRouteSpec,
 			},
 		},
-		want: apis.ErrInvalidValue("bad-label", "metadata.labels.serving.knative.dev/visibility"),
+		want: apis.ErrInvalidValue("bad-label", "metadata.labels."+network.VisibilityLabelKey),
 	}, {
 		name: "valid release",
 		r: &Service{
diff --git a/pkg/autoscaler/statforwarder/forwarder_test.go b/pkg/autoscaler/statforwarder/forwarder_test.go
index d2a007f54..9ddeb5a5b 100644
--- a/pkg/autoscaler/statforwarder/forwarder_test.go
+++ b/pkg/autoscaler/statforwarder/forwarder_test.go
@@ -79,14 +79,16 @@ func TestForwarderReconcile(t *testing.T) {
 		t.Fatal("Failed to start informers:", err)
 	}
 
+	f1 := New(ctx, logger, kubeClient, testIP1, testBs, noOp)
+	f2 := New(ctx, logger, kubeClient, testIP2, testBs, noOp)
+
 	defer func() {
+		f1.Cancel()
+		f2.Cancel()
 		cancel()
 		waitInformers()
 	}()
 
-	New(ctx, logger, kubeClient, testIP1, testBs, noOp)
-	New(ctx, logger, kubeClient, testIP2, testBs, noOp)
-
 	kubeClient.CoordinationV1().Leases(testNs).Create(testLease)
 	lease.Informer().GetIndexer().Add(testLease)
 
diff --git a/pkg/reconciler/revision/resolve_test.go b/pkg/reconciler/revision/resolve_test.go
index e0442d58a..82a8dbb78 100644
--- a/pkg/reconciler/revision/resolve_test.go
+++ b/pkg/reconciler/revision/resolve_test.go
@@ -65,27 +65,28 @@ func mustRawManifest(t *testing.T, img v1.Image) []byte {
 
 func fakeRegistry(t *testing.T, repo, username, password string, img v1.Image) *httptest.Server {
 	manifestPath := fmt.Sprintf("/v2/%s/manifests/latest", repo)
+	const basicAuth = "Basic "
 	return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		switch r.URL.Path {
 		case "/v2/":
 			// Issue a "Basic" auth challenge, so we can check the auth sent to the registry.
-			w.Header().Set("WWW-Authenticate", `Basic `)
+			w.Header().Set("WWW-Authenticate", basicAuth)
 			http.Error(w, "Unauthorized", http.StatusUnauthorized)
 		case manifestPath:
 			// Check that we get an auth header with base64 encoded username:password
 			hdr := r.Header.Get("Authorization")
-			if !strings.HasPrefix(hdr, "Basic ") {
-				t.Errorf("Header.Get(Authorization); got %v, want Basic prefix", hdr)
+			if !strings.HasPrefix(hdr, basicAuth) {
+				t.Errorf("Header.Get(Authorization) = %q, want %q prefix", hdr, basicAuth)
 			}
 			if want := base64.StdEncoding.EncodeToString([]byte(username + ":" + password)); !strings.HasSuffix(hdr, want) {
-				t.Errorf("Header.Get(Authorization); got %v, want suffix %v", hdr, want)
+				t.Errorf("Header.Get(Authorization) = %q, want suffix %q", hdr, want)
 			}
 			if r.Method != http.MethodGet {
-				t.Errorf("Method; got %v, want %v", r.Method, http.MethodGet)
+				t.Errorf("Method = %v, want %v", r.Method, http.MethodGet)
 			}
 			w.Write(mustRawManifest(t, img))
 		default:
-			t.Fatalf("Unexpected path: %v", r.URL.Path)
+			t.Error("Unexpected path:", r.URL.Path)
 		}
 	}))
 }
@@ -96,7 +97,7 @@ func fakeRegistryPingFailure(t *testing.T) *httptest.Server {
 		case "/v2/":
 			http.Error(w, "Oops", http.StatusInternalServerError)
 		default:
-			t.Fatalf("Unexpected path: %v", r.URL.Path)
+			t.Error("Unexpected path:", r.URL.Path)
 		}
 	}))
 }
@@ -112,7 +113,7 @@ func fakeRegistryManifestFailure(t *testing.T, repo string) *httptest.Server {
 		case manifestPath:
 			http.Error(w, "Boom", http.StatusInternalServerError)
 		default:
-			t.Fatalf("Unexpected path: %v", r.URL.Path)
+			t.Error("Unexpected path:", r.URL.Path)
 		}
 	}))
 }
@@ -150,7 +151,7 @@ func TestResolve(t *testing.T) {
 		t.Fatalf("random.Image() = %v", err)
 	}
 
-	// Stand up a fake registry
+	// Stand up a fake registry.
 	server := fakeRegistry(t, expectedRepo, username, password, img)
 	defer server.Close()
 	u, err := url.Parse(server.URL)
@@ -158,13 +159,13 @@ func TestResolve(t *testing.T) {
 		t.Fatal("url.Parse() =", err)
 	}
 
-	// Create a tag pointing to an image on our fake registry
+	// Create a tag pointing to an image on our fake registry.
 	tag, err := name.NewTag(fmt.Sprintf("%s/%s:latest", u.Host, expectedRepo), name.WeakValidation)
 	if err != nil {
-		t.Fatalf("NewTag() = %v", err)
+		t.Fatal("NewTag() =", err)
 	}
 
-	// Set up a fake service account with pull secrets for our fake registry
+	// Set up a fake service account with pull secrets for our fake registry.
 	client := fakeclient.NewSimpleClientset(&corev1.ServiceAccount{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      svcacct,
@@ -187,7 +188,7 @@ func TestResolve(t *testing.T) {
 		},
 	})
 
-	// Resolve our tag on the fake registry to the digest of the random.Image()
+	// Resolve our tag on the fake registry to the digest of the random.Image().
 	dr := &digestResolver{client: client, transport: http.DefaultTransport}
 	opt := k8schain.Options{
 		Namespace:          ns,
@@ -195,13 +196,13 @@ func TestResolve(t *testing.T) {
 	}
 	resolvedDigest, err := dr.Resolve(context.Background(), tag.String(), opt, emptyRegistrySet)
 	if err != nil {
-		t.Fatalf("Resolve() = %v", err)
+		t.Fatal("Resolve() =", err)
 	}
 
 	// Make sure that we get back the appropriate digest.
 	digest, err := name.NewDigest(resolvedDigest, name.WeakValidation)
 	if err != nil {
-		t.Fatalf("NewDigest() = %v", err)
+		t.Fatal("NewDigest() =", err)
 	}
 	if got, want := digest.DigestStr(), mustDigest(t, img).String(); got != want {
 		t.Fatalf("Resolve() = %v, want %v", got, want)
@@ -231,15 +232,18 @@ func TestResolveWithDigest(t *testing.T) {
 	}
 
 	if diff := cmp.Diff(originalDigest, resolvedDigest); diff != "" {
-		t.Errorf("Digest should not change (-want +got): %s", diff)
+		t.Errorf("Digest should not change (-want +got):\n%s", diff)
 	}
 }
 
 func TestResolveWithBadTag(t *testing.T) {
-	ns, svcacct := "foo", "default"
+	const (
+		ns      = "foo"
+		svcacct = "default"
+	)
 	client := fakeclient.NewSimpleClientset(&corev1.ServiceAccount{
 		ObjectMeta: metav1.ObjectMeta{
-			Name:      "default",
+			Name:      svcacct,
 			Namespace: ns,
 		},
 	})
@@ -250,10 +254,10 @@ func TestResolveWithBadTag(t *testing.T) {
 		ServiceAccountName: svcacct,
 	}
 
-	// Invalid character
+	// Invalid character.
 	invalidImage := "ubuntu%latest"
 	if resolvedDigest, err := dr.Resolve(context.Background(), invalidImage, opt, emptyRegistrySet); err == nil {
-		t.Fatalf("Resolve() = %v, want error", resolvedDigest)
+		t.Fatalf("Resolve() succeeded with %q, want error", resolvedDigest)
 	}
 }
 
diff --git a/pkg/reconciler/route/domains/domains_test.go b/pkg/reconciler/route/domains/domains_test.go
index a609ed859..65a1b5cb7 100644
--- a/pkg/reconciler/route/domains/domains_test.go
+++ b/pkg/reconciler/route/domains/domains_test.go
@@ -135,9 +135,9 @@ func TestDomainNameFromTemplate(t *testing.T) {
 			ctx = config.ToContext(ctx, cfg)
 
 			if tt.local {
-				meta.Labels[serving.VisibilityLabelKey] = serving.VisibilityClusterLocal
+				meta.Labels[network.VisibilityLabelKey] = serving.VisibilityClusterLocal
 			} else {
-				delete(meta.Labels, serving.VisibilityLabelKey)
+				delete(meta.Labels, network.VisibilityLabelKey)
 			}
 
 			got, err := DomainNameFromTemplate(ctx, meta, tt.args.name)
diff --git a/pkg/reconciler/route/resources/filters_test.go b/pkg/reconciler/route/resources/filters_test.go
index 0e0b7a898..7162b2f6b 100644
--- a/pkg/reconciler/route/resources/filters_test.go
+++ b/pkg/reconciler/route/resources/filters_test.go
@@ -22,6 +22,7 @@ import (
 
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 )
 
 func TestIsClusterLocalService(t *testing.T) {
@@ -37,7 +38,7 @@ func TestIsClusterLocalService(t *testing.T) {
 		svc: &corev1.Service{
 			ObjectMeta: metav1.ObjectMeta{
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: "something-unknown",
+					network.VisibilityLabelKey: "something-unknown",
 				},
 			},
 		},
@@ -46,7 +47,7 @@ func TestIsClusterLocalService(t *testing.T) {
 		svc: &corev1.Service{
 			ObjectMeta: metav1.ObjectMeta{
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		},
diff --git a/pkg/reconciler/route/resources/labels/labels_test.go b/pkg/reconciler/route/resources/labels/labels_test.go
index a5f8a68bd..0e3363108 100644
--- a/pkg/reconciler/route/resources/labels/labels_test.go
+++ b/pkg/reconciler/route/resources/labels/labels_test.go
@@ -21,6 +21,7 @@ import (
 
 	"github.com/google/go-cmp/cmp"
 	v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	network "knative.dev/networking/pkg"
 	"knative.dev/serving/pkg/apis/serving"
 )
 
@@ -45,12 +46,12 @@ func TestIsObjectLocalVisibility(t *testing.T) {
 	}, {
 		name: "false",
 		meta: &v1.ObjectMeta{
-			Labels: map[string]string{serving.VisibilityLabelKey: ""},
+			Labels: map[string]string{network.VisibilityLabelKey: ""},
 		},
 	}, {
 		name: "true",
 		meta: &v1.ObjectMeta{
-			Labels: map[string]string{serving.VisibilityLabelKey: "set"},
+			Labels: map[string]string{network.VisibilityLabelKey: "set"},
 		},
 		want: true,
 	}}
@@ -164,10 +165,10 @@ func TestSetVisibility(t *testing.T) {
 		name:           "Set cluster local true",
 		meta:           &v1.ObjectMeta{},
 		isClusterLocal: true,
-		want:           v1.ObjectMeta{Labels: map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal}},
+		want:           v1.ObjectMeta{Labels: map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal}},
 	}, {
 		name: "Set cluster local false",
-		meta: &v1.ObjectMeta{Labels: map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal}},
+		meta: &v1.ObjectMeta{Labels: map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal}},
 		want: v1.ObjectMeta{Labels: map[string]string{}},
 	}}
 	for _, tt := range tests {
diff --git a/pkg/reconciler/route/resources/service_test.go b/pkg/reconciler/route/resources/service_test.go
index 347ae83db..b67c8e6b2 100644
--- a/pkg/reconciler/route/resources/service_test.go
+++ b/pkg/reconciler/route/resources/service_test.go
@@ -309,7 +309,7 @@ func TestMakeK8sPlaceholderService(t *testing.T) {
 		wantErr: false,
 	}, {
 		name:  "cluster local route",
-		route: Route("test-ns", "test-route", WithRouteLabel(map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal})),
+		route: Route("test-ns", "test-route", WithRouteLabel(map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal})),
 		expectedSpec: corev1.ServiceSpec{
 			Type:            corev1.ServiceTypeExternalName,
 			ExternalName:    "foo-test-route.test-ns.svc.cluster.local",
diff --git a/pkg/reconciler/route/table_test.go b/pkg/reconciler/route/table_test.go
index db5224bf4..b8585076c 100644
--- a/pkg/reconciler/route/table_test.go
+++ b/pkg/reconciler/route/table_test.go
@@ -281,7 +281,7 @@ func TestReconcile(t *testing.T) {
 		Name: "cluster local route becomes ready, ingress unknown",
 		Objects: []runtime.Object{
 			Route("default", "becomes-ready", WithConfigTarget("config"), WithLocalDomain,
-				WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"}),
 				WithRouteUID("65-23"), WithRouteGeneration(1)),
 			cfg("default", "config",
 				WithConfigGeneration(1), WithLatestCreated("config-00001"), WithLatestReady("config-00001")),
@@ -291,7 +291,7 @@ func TestReconcile(t *testing.T) {
 			simpleIngress(
 				Route("default", "becomes-ready", WithConfigTarget("config"),
 					WithLocalDomain, WithRouteUID("65-23"),
-					WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"})),
+					WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -312,7 +312,7 @@ func TestReconcile(t *testing.T) {
 			simplePlaceholderK8sService(
 				getContext(),
 				Route("default", "becomes-ready", WithConfigTarget("config"), WithLocalDomain,
-					WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"}),
+					WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"}),
 					WithRouteUID("65-23"), WithRouteGeneration(1)),
 				"",
 			),
@@ -322,7 +322,7 @@ func TestReconcile(t *testing.T) {
 				WithRouteUID("65-23"), WithRouteGeneration(1), WithRouteObservedGeneration,
 				// Populated by reconciliation when all traffic has been assigned.
 				WithLocalDomain, WithAddress, WithRouteConditionsAutoTLSDisabled,
-				WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"}),
 				MarkTrafficAssigned, MarkIngressNotConfigured, WithStatusTraffic(
 					v1.TrafficTarget{
 						RevisionName:   "config-00001",
@@ -721,13 +721,13 @@ func TestReconcile(t *testing.T) {
 		Name: "public becomes cluster local",
 		Objects: []runtime.Object{
 			Route("default", "becomes-local", WithConfigTarget("config"), WithRouteGeneration(1),
-				WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"}),
 				WithRouteUID("65-23")),
 			cfg("default", "config",
 				WithConfigGeneration(1), WithLatestCreated("config-00001"), WithLatestReady("config-00001")),
 			rev("default", "config", 1, MarkRevisionReady, WithRevName("config-00001"), WithServiceName("tb")),
 			simpleIngress(
-				Route("default", "becomes-local", WithConfigTarget("config"), WithRouteUID("65-23"), WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"})),
+				Route("default", "becomes-local", WithConfigTarget("config"), WithRouteUID("65-23"), WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -743,14 +743,14 @@ func TestReconcile(t *testing.T) {
 				},
 			),
 			simpleK8sService(Route("default", "becomes-local", WithConfigTarget("config"),
-				WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"}),
 				WithRouteUID("65-23"))),
 		},
 		WantUpdates: []clientgotesting.UpdateActionImpl{{
 			Object: simpleIngress(
 				Route("default", "becomes-local", WithConfigTarget("config"), WithRouteGeneration(1),
 					WithRouteUID("65-23"),
-					WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"})),
+					WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -774,7 +774,7 @@ func TestReconcile(t *testing.T) {
 				WithRouteUID("65-23"), WithRouteGeneration(1), WithRouteObservedGeneration,
 				MarkTrafficAssigned, MarkIngressNotConfigured,
 				WithLocalDomain, WithAddress, WithRouteConditionsAutoTLSDisabled,
-				WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"}),
 				WithStatusTraffic(
 					v1.TrafficTarget{
 						RevisionName:   "config-00001",
@@ -793,7 +793,7 @@ func TestReconcile(t *testing.T) {
 			rev("default", "config", 1, MarkRevisionReady, WithRevName("config-00001"), WithServiceName("tb")),
 			simpleIngress(
 				Route("default", "becomes-public", WithConfigTarget("config"), WithRouteUID("65-23"),
-					WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"})),
+					WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -818,7 +818,7 @@ func TestReconcile(t *testing.T) {
 			Object: simpleIngress(
 				Route("default", "becomes-public", WithConfigTarget("config"),
 					WithRouteUID("65-23"), WithRouteGeneration(1), WithRouteObservedGeneration,
-					WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"})),
+					WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -2402,13 +2402,13 @@ func TestReconcileEnableAutoTLS(t *testing.T) {
 		Name: "public becomes cluster local w/ autoTLS",
 		Objects: []runtime.Object{
 			Route("default", "becomes-local", WithConfigTarget("config"), WithRouteGeneration(1),
-				WithRouteLabel(map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal}),
 				WithRouteUID("65-23")),
 			cfg("default", "config",
 				WithConfigGeneration(1), WithLatestCreated("config-00001"), WithLatestReady("config-00001")),
 			rev("default", "config", 1, MarkRevisionReady, WithRevName("config-00001"), WithServiceName("tb")),
 			simpleIngress(
-				Route("default", "becomes-local", WithConfigTarget("config"), WithRouteUID("65-23"), WithRouteLabel(map[string]string{"serving.knative.dev/visibility": "cluster-local"})),
+				Route("default", "becomes-local", WithConfigTarget("config"), WithRouteUID("65-23"), WithRouteLabel(map[string]string{network.VisibilityLabelKey: "cluster-local"})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -2424,14 +2424,14 @@ func TestReconcileEnableAutoTLS(t *testing.T) {
 				},
 			),
 			simpleK8sService(Route("default", "becomes-local", WithConfigTarget("config"),
-				WithRouteLabel(map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal}),
 				WithRouteUID("65-23"))),
 		},
 		WantUpdates: []clientgotesting.UpdateActionImpl{{
 			Object: simpleIngress(
 				Route("default", "becomes-local", WithConfigTarget("config"),
 					WithRouteUID("65-23"), WithRouteGeneration(1), WithRouteObservedGeneration,
-					WithRouteLabel(map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal})),
+					WithRouteLabel(map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal})),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
 						traffic.DefaultTarget: {{
@@ -2456,7 +2456,7 @@ func TestReconcileEnableAutoTLS(t *testing.T) {
 				WithRouteGeneration(1), WithRouteObservedGeneration,
 				MarkTrafficAssigned, MarkIngressNotConfigured, WithRouteConditionsTLSNotEnabledForClusterLocalMessage,
 				WithLocalDomain, WithAddress, WithInitRouteConditions,
-				WithRouteLabel(map[string]string{serving.VisibilityLabelKey: serving.VisibilityClusterLocal}),
+				WithRouteLabel(map[string]string{network.VisibilityLabelKey: serving.VisibilityClusterLocal}),
 				WithStatusTraffic(
 					v1.TrafficTarget{
 						RevisionName:   "config-00001",
diff --git a/pkg/reconciler/route/visibility/visibility_test.go b/pkg/reconciler/route/visibility/visibility_test.go
index 40335d243..0a90803e2 100644
--- a/pkg/reconciler/route/visibility/visibility_test.go
+++ b/pkg/reconciler/route/visibility/visibility_test.go
@@ -77,7 +77,7 @@ func TestVisibility(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "foo",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		},
@@ -96,7 +96,7 @@ func TestVisibility(t *testing.T) {
 				Name: "foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}, {
@@ -104,7 +104,7 @@ func TestVisibility(t *testing.T) {
 				Name: "irrelevance",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "bar",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}},
@@ -126,7 +126,7 @@ func TestVisibility(t *testing.T) {
 				Name: "blue-foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}, {
@@ -268,7 +268,7 @@ func TestVisibility(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "foo",
 				Labels: map[string]string{
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 			Spec: v1.RouteSpec{
@@ -323,7 +323,7 @@ func TestVisibility(t *testing.T) {
 				Name: "blue-foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}},
@@ -350,7 +350,7 @@ func TestVisibility(t *testing.T) {
 				Name: "blue-foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}, {
@@ -358,7 +358,7 @@ func TestVisibility(t *testing.T) {
 				Name: "green-foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}},
@@ -385,7 +385,7 @@ func TestVisibility(t *testing.T) {
 				Name: "blue-foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}, {
@@ -393,7 +393,7 @@ func TestVisibility(t *testing.T) {
 				Name: "green-foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}, {
@@ -401,7 +401,7 @@ func TestVisibility(t *testing.T) {
 				Name: "foo",
 				Labels: map[string]string{
 					serving.RouteLabelKey:      "foo",
-					serving.VisibilityLabelKey: serving.VisibilityClusterLocal,
+					network.VisibilityLabelKey: serving.VisibilityClusterLocal,
 				},
 			},
 		}},
diff --git a/test/e2e/route_service_test.go b/test/e2e/route_service_test.go
index 1154f283f..29608e154 100644
--- a/test/e2e/route_service_test.go
+++ b/test/e2e/route_service_test.go
@@ -22,6 +22,7 @@ import (
 	"strings"
 	"testing"
 
+	netpkg "knative.dev/networking/pkg"
 	"knative.dev/pkg/network"
 	"knative.dev/pkg/ptr"
 	v1 "knative.dev/serving/pkg/apis/serving/v1"
@@ -162,7 +163,7 @@ func TestRouteVisibilityChanges(t *testing.T) {
 			}
 
 			v1test.PatchService(st, clients, svc, func(s *v1.Service) {
-				s.SetLabels(map[string]string{"serving.knative.dev/visibility": "cluster-local"})
+				s.SetLabels(map[string]string{netpkg.VisibilityLabelKey: "cluster-local"})
 			})
 
 			st.Logf("Waiting for Service %q ObservedGeneration to match Generation, and status transition to Ready == True", names.Service)
diff --git a/test/e2e/service_to_service_test.go b/test/e2e/service_to_service_test.go
index f2df2f5cf..64d84671f 100644
--- a/test/e2e/service_to_service_test.go
+++ b/test/e2e/service_to_service_test.go
@@ -25,6 +25,7 @@ import (
 	"testing"
 
 	corev1 "k8s.io/api/core/v1"
+	network "knative.dev/networking/pkg"
 	pkgTest "knative.dev/pkg/test"
 	ingress "knative.dev/pkg/test/ingress"
 	"knative.dev/pkg/test/logstream"
@@ -180,7 +181,7 @@ func TestServiceToServiceCall(t *testing.T) {
 	test.EnsureTearDown(t, clients, &names)
 
 	withInternalVisibility := rtesting.WithServiceLabel(
-		serving.VisibilityLabelKey, serving.VisibilityClusterLocal)
+		network.VisibilityLabelKey, serving.VisibilityClusterLocal)
 	resources, err := v1test.CreateServiceReady(t, clients, &names, withInternalVisibility)
 	if err != nil {
 		t.Fatalf("Failed to create initial Service: %v: %v", names.Service, err)
@@ -222,7 +223,7 @@ func testSvcToSvcCallViaActivator(t *testing.T, clients *test.Clients, injectA b
 	}
 
 	withInternalVisibility := rtesting.WithServiceLabel(
-		serving.VisibilityLabelKey, serving.VisibilityClusterLocal)
+		network.VisibilityLabelKey, serving.VisibilityClusterLocal)
 
 	test.EnsureTearDown(t, clients, &testNames)
 
diff --git a/test/e2e/subroutes_test.go b/test/e2e/subroutes_test.go
index 150418749..86e92c1a0 100644
--- a/test/e2e/subroutes_test.go
+++ b/test/e2e/subroutes_test.go
@@ -26,6 +26,7 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/types"
 
+	netpkg "knative.dev/networking/pkg"
 	"knative.dev/pkg/apis/duck"
 	"knative.dev/pkg/network"
 	"knative.dev/pkg/ptr"
@@ -58,7 +59,7 @@ func TestSubrouteLocalSTS(t *testing.T) { // We can't use a longer more descript
 
 	tag := "current"
 
-	withInternalVisibility := rtesting.WithServiceLabel(serving.VisibilityLabelKey, serving.VisibilityClusterLocal)
+	withInternalVisibility := rtesting.WithServiceLabel(netpkg.VisibilityLabelKey, serving.VisibilityClusterLocal)
 	withTrafficSpec := rtesting.WithRouteSpec(v1.RouteSpec{
 		Traffic: []v1.TrafficTarget{
 			{
@@ -215,7 +216,7 @@ func TestSubrouteVisibilityPrivateToPublic(t *testing.T) {
 	subrouteTag1 := "my-tag"
 	subrouteTag2 := "my-tag2"
 
-	withInternalVisibility := rtesting.WithServiceLabel(serving.VisibilityLabelKey, serving.VisibilityClusterLocal)
+	withInternalVisibility := rtesting.WithServiceLabel(netpkg.VisibilityLabelKey, serving.VisibilityClusterLocal)
 	withTrafficSpec := rtesting.WithRouteSpec(v1.RouteSpec{
 		Traffic: []v1.TrafficTarget{
 			{