Repository: knative/serving
Pull request #10329 (2020-12-09 14:55:01)

Split apart the forwarder.
The forwarder currently conflates two separable things:
1. Resolving the current owner of a bucket.
2. Forwarding stats to the owner of a bucket.

This builds on https://github.com/knative/serving/pull/10313 and splits the lease tracking logic off of Forwarder.

Ultimately the goals (not realized here!) of this are multi-fold:
1. The forwarder doesn't understand anything about leases or bucket anatomy, it uses an "oracle" of sorts to tell it where to send things.
2. The lease tracker implements the "oracle" functionality above by decoding lease holder identities.
3. As a proof-point of the abstraction, we can plug in StatefulSets as an alternative bucketing mechanism (and forgo LE entirely).


knative-prow-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/knative/serving/pull/10329#" title="Author self-approved">mattmoor</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=knative%2Fserving).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[pkg/autoscaler/OWNERS](https://github.com/knative/serving/blob/master/pkg/autoscaler/OWNERS)~~ [mattmoor]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
mattmoor: The goal of this is splitting functionality across two structs, so nothing meaningful should change here other than better equipping us to move these two things further apart behind stronger contracts.
codecov[bot]: # [Codecov](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=h1) Report
> Merging [#10329](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=desc) (ebed299) into [master](https://codecov.io/gh/knative/serving/commit/2739e2d6071d813831764b803927c3c08a819873?el=desc) (2739e2d) will **increase** coverage by `0.04%`.
> The diff coverage is `95.55%`.

[![Impacted file tree graph](https://codecov.io/gh/knative/serving/pull/10329/graphs/tree.svg?width=650&height=150&src=pr&token=MPpDfAj4Ui)](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #10329      +/-   ##
==========================================
+ Coverage   88.21%   88.25%   +0.04%     
==========================================
  Files         184      184              
  Lines        8660     8677      +17     
==========================================
+ Hits         7639     7658      +19     
+ Misses        786      785       -1     
+ Partials      235      234       -1     
```


| [Impacted Files](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pkg/autoscaler/statforwarder/forwarder.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL2F1dG9zY2FsZXIvc3RhdGZvcndhcmRlci9mb3J3YXJkZXIuZ28=) | `83.06% <93.54%> (+1.68%)` | :arrow_up: |
| [pkg/autoscaler/statforwarder/processor.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL2F1dG9zY2FsZXIvc3RhdGZvcndhcmRlci9wcm9jZXNzb3IuZ28=) | `88.88% <100.00%> (+5.55%)` | :arrow_up: |
| [pkg/reconciler/revision/reconcile\_resources.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL3JlY29uY2lsZXIvcmV2aXNpb24vcmVjb25jaWxlX3Jlc291cmNlcy5nbw==) | `80.72% <0.00%> (-2.41%)` | :arrow_down: |
| [pkg/autoscaler/scaling/multiscaler.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL2F1dG9zY2FsZXIvc2NhbGluZy9tdWx0aXNjYWxlci5nbw==) | `87.93% <0.00%> (-1.73%)` | :arrow_down: |
| [pkg/apis/serving/v1alpha1/domainmapping\_types.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL2FwaXMvc2VydmluZy92MWFscGhhMS9kb21haW5tYXBwaW5nX3R5cGVzLmdv) | `100.00% <0.00%> (ø)` | |
| [pkg/reconciler/domainmapping/reconciler.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL3JlY29uY2lsZXIvZG9tYWlubWFwcGluZy9yZWNvbmNpbGVyLmdv) | `89.33% <0.00%> (+0.14%)` | :arrow_up: |
| [pkg/activator/net/revision\_backends.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL2FjdGl2YXRvci9uZXQvcmV2aXNpb25fYmFja2VuZHMuZ28=) | `91.40% <0.00%> (+0.90%)` | :arrow_up: |
| [...g/apis/serving/v1alpha1/domainmapping\_lifecycle.go](https://codecov.io/gh/knative/serving/pull/10329/diff?src=pr&el=tree#diff-cGtnL2FwaXMvc2VydmluZy92MWFscGhhMS9kb21haW5tYXBwaW5nX2xpZmVjeWNsZS5nbw==) | `85.71% <0.00%> (+4.23%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=footer). Last update [2739e2d...ebed299](https://codecov.io/gh/knative/serving/pull/10329?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/.github/workflows/knative-boilerplate.yaml b/.github/workflows/knative-boilerplate.yaml
index 9329fdd7e..c0e648b1c 100644
--- a/.github/workflows/knative-boilerplate.yaml
+++ b/.github/workflows/knative-boilerplate.yaml
@@ -19,7 +19,7 @@ name: Boilerplate
 
 on:
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
 
diff --git a/.github/workflows/knative-donotsubmit.yaml b/.github/workflows/knative-donotsubmit.yaml
index d9131173d..cec5c3077 100644
--- a/.github/workflows/knative-donotsubmit.yaml
+++ b/.github/workflows/knative-donotsubmit.yaml
@@ -19,7 +19,7 @@ name: Do Not Submit
 
 on:
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
 
diff --git a/.github/workflows/knative-go-build.yaml b/.github/workflows/knative-go-build.yaml
index 226eac87d..7bd13004a 100644
--- a/.github/workflows/knative-go-build.yaml
+++ b/.github/workflows/knative-go-build.yaml
@@ -19,7 +19,7 @@ name: Build
 
 on:
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
 
diff --git a/.github/workflows/knative-security.yaml b/.github/workflows/knative-security.yaml
index 47186ef2e..00fa9fccf 100644
--- a/.github/workflows/knative-security.yaml
+++ b/.github/workflows/knative-security.yaml
@@ -19,10 +19,10 @@ name: 'Security'
 
 on:
   push:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
   analyze:
diff --git a/.github/workflows/knative-style.yaml b/.github/workflows/knative-style.yaml
index 5d3d3163e..b38122ab6 100644
--- a/.github/workflows/knative-style.yaml
+++ b/.github/workflows/knative-style.yaml
@@ -19,7 +19,7 @@ name: Code Style
 
 on:
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
 
diff --git a/.github/workflows/knative-verify.yaml b/.github/workflows/knative-verify.yaml
index ed5bb3a13..3443a0496 100644
--- a/.github/workflows/knative-verify.yaml
+++ b/.github/workflows/knative-verify.yaml
@@ -19,7 +19,7 @@ name: Verify
 
 on:
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
 
diff --git a/go.mod b/go.mod
index 56fb7a444..6eac9dd7a 100644
--- a/go.mod
+++ b/go.mod
@@ -36,6 +36,6 @@ require (
 	k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29
 	knative.dev/caching v0.0.0-20201208013740-3ca9791186ed
 	knative.dev/hack v0.0.0-20201201234937-fddbf732e450
-	knative.dev/networking v0.0.0-20201208014140-4d1d53eb0626
-	knative.dev/pkg v0.0.0-20201209011241-b1ac587fdd80
+	knative.dev/networking v0.0.0-20201209014241-22a7e22628fc
+	knative.dev/pkg v0.0.0-20201209172941-9b9407a76df1
 )
diff --git a/go.sum b/go.sum
index 7c081bb47..2f036d511 100644
--- a/go.sum
+++ b/go.sum
@@ -1220,11 +1220,13 @@ knative.dev/caching v0.0.0-20201208013740-3ca9791186ed h1:Hl6jZZFVPRv++MDHtD9hy8
 knative.dev/caching v0.0.0-20201208013740-3ca9791186ed/go.mod h1:i+e4gaoBB34anvKMMiQDV86N7HbFYN6m+Op1IRhXXY8=
 knative.dev/hack v0.0.0-20201201234937-fddbf732e450 h1:IyitWF7OzfunCgE4b9ZsJAeIRoQ6JOyqDrz/19X4iS4=
 knative.dev/hack v0.0.0-20201201234937-fddbf732e450/go.mod h1:PHt8x8yX5Z9pPquBEfIj0X66f8iWkWfR0S/sarACJrI=
-knative.dev/networking v0.0.0-20201208014140-4d1d53eb0626 h1:e8IIJxggzVD7kMNf+4mYY421vcWpD2U1jXXIRDWy7bg=
-knative.dev/networking v0.0.0-20201208014140-4d1d53eb0626/go.mod h1:V4CU4mLLkWboLw3NOVmPHHOCL5nrUO3/vsRDUjsnJUQ=
+knative.dev/networking v0.0.0-20201209014241-22a7e22628fc h1:Zav6xAGiSKddU2ra70oVgYkB3KjJUpiTqi7A5iAaUoE=
+knative.dev/networking v0.0.0-20201209014241-22a7e22628fc/go.mod h1:NPlekJSE5w2vw5mcu8zCBdHsiBq9U/waDbFXrEvpw6Q=
 knative.dev/pkg v0.0.0-20201207225040-7e918163fc4f/go.mod h1:Jtbz2/1gB8EZgZA4VvabTxM+AcDt2WL2P0RaFnKcOX8=
 knative.dev/pkg v0.0.0-20201209011241-b1ac587fdd80 h1:wBRwTtE3saOiXsS/eh4FYDAC/Gj+djH8ZNkvd0OgtmE=
 knative.dev/pkg v0.0.0-20201209011241-b1ac587fdd80/go.mod h1:Jtbz2/1gB8EZgZA4VvabTxM+AcDt2WL2P0RaFnKcOX8=
+knative.dev/pkg v0.0.0-20201209172941-9b9407a76df1 h1:whfizV9at+XvlOK66xm1WfiiPhybrRa9LbZa4uXbWIs=
+knative.dev/pkg v0.0.0-20201209172941-9b9407a76df1/go.mod h1:Jtbz2/1gB8EZgZA4VvabTxM+AcDt2WL2P0RaFnKcOX8=
 modernc.org/cc v1.0.0/go.mod h1:1Sk4//wdnYJiUIxnW8ddKpaOJCF37yAdqYnkxUpaYxw=
 modernc.org/golex v1.0.0/go.mod h1:b/QX9oBD/LhixY6NDh+IdGv17hgB+51fET1i2kPSmvk=
 modernc.org/mathutil v1.0.0/go.mod h1:wU0vUrJsVWBZ4P6e7xtFJEhFSNsfRLJ8H458uRjg03k=
diff --git a/pkg/autoscaler/statforwarder/forwarder.go b/pkg/autoscaler/statforwarder/forwarder.go
index 1d3ed2216..30efe1782 100644
--- a/pkg/autoscaler/statforwarder/forwarder.go
+++ b/pkg/autoscaler/statforwarder/forwarder.go
@@ -73,16 +73,10 @@ type stat struct {
 // 3. Can be used to forward the metrics owned by a bucket based on
 //    the holder IP.
 type Forwarder struct {
-	selfIP          string
-	logger          *zap.SugaredLogger
-	kc              kubernetes.Interface
-	endpointsLister corev1listers.EndpointsLister
-
-	// `accept` is the function to process a StatMessage which doesn't need
-	// to be forwarded.
-	accept statProcessor
+	logger *zap.SugaredLogger
 	// bs is the BucketSet including all Autoscaler buckets.
 	bs *hash.BucketSet
+
 	// processorsLock is the lock for processors.
 	processorsLock sync.RWMutex
 	processors     map[string]bucketProcessor
@@ -93,41 +87,42 @@ type Forwarder struct {
 	// on when shutting down.
 	processingWg sync.WaitGroup
 
-	// id2ip stores the IP extracted from the holder identity to avoid
-	// string split each time.
-	id2ip map[string]string
-
 	statCh chan stat
 	stopCh chan struct{}
 }
 
 // New creates a new Forwarder.
 func New(ctx context.Context, logger *zap.SugaredLogger, kc kubernetes.Interface, selfIP string, bs *hash.BucketSet, accept statProcessor) *Forwarder {
-	ns := system.Namespace()
 	bkts := bs.Buckets()
 	endpointsInformer := endpointsinformer.Get(ctx)
 	f := &Forwarder{
-		selfIP:          selfIP,
+		logger:     logger,
+		bs:         bs,
+		processors: make(map[string]bucketProcessor, len(bkts)),
+		statCh:     make(chan stat, 1000),
+		stopCh:     make(chan struct{}),
+	}
+
+	f.processingWg.Add(1)
+	go f.process()
+
+	lt := &leaseTracker{
 		logger:          logger,
+		selfIP:          selfIP,
+		bs:              bs,
 		kc:              kc,
 		endpointsLister: endpointsInformer.Lister(),
-		bs:              bs,
-		processors:      make(map[string]bucketProcessor, len(bkts)),
-		accept:          accept,
 		id2ip:           make(map[string]string),
-		statCh:          make(chan stat, 1000),
-		stopCh:          make(chan struct{}),
+		accept:          accept,
+		fwd:             f,
 	}
 
-	f.processingWg.Add(1)
-	go f.process()
-
 	leaseInformer := leaseinformer.Get(ctx)
 	leaseInformer.Informer().AddEventHandler(cache.FilteringResourceEventHandler{
-		FilterFunc: f.filterFunc(ns),
+		FilterFunc: lt.filterFunc(system.Namespace()),
 		Handler: cache.ResourceEventHandlerFuncs{
-			AddFunc:    f.leaseUpdated,
-			UpdateFunc: controller.PassNew(f.leaseUpdated),
+			AddFunc:    lt.leaseUpdated,
+			UpdateFunc: controller.PassNew(lt.leaseUpdated),
 			// TODO(yanweiguo): Set up DeleteFunc.
 		},
 	})
@@ -135,7 +130,31 @@ func New(ctx context.Context, logger *zap.SugaredLogger, kc kubernetes.Interface
 	return f
 }
 
-func (f *Forwarder) extractIP(holder string) (string, error) {
+// leaseTracker monitors lease resources to update the Forwarder's processor configuration(s)
+// with the appropriate owner's stats endpoint.  When we own the lease, a localProcessor is
+// used, and when someone else owns the lease a remoteProcessor is used with their stats
+// endpoint.
+type leaseTracker struct {
+	logger *zap.SugaredLogger
+	selfIP string
+	// bs is the BucketSet including all Autoscaler buckets.
+	bs *hash.BucketSet
+
+	kc              kubernetes.Interface
+	endpointsLister corev1listers.EndpointsLister
+
+	// id2ip stores the IP extracted from the holder identity to avoid
+	// string split each time.
+	id2ip map[string]string
+
+	// `accept` is the function to process a StatMessage which doesn't need
+	// to be forwarded.
+	accept statProcessor
+
+	fwd *Forwarder
+}
+
+func (f *leaseTracker) extractIP(holder string) (string, error) {
 	if ip, ok := f.id2ip[holder]; ok {
 		return ip, nil
 	}
@@ -147,7 +166,7 @@ func (f *Forwarder) extractIP(holder string) (string, error) {
 	return ip, err
 }
 
-func (f *Forwarder) filterFunc(namespace string) func(interface{}) bool {
+func (f *leaseTracker) filterFunc(namespace string) func(interface{}) bool {
 	return func(obj interface{}) bool {
 		l := obj.(*coordinationv1.Lease)
 
@@ -172,7 +191,7 @@ func (f *Forwarder) filterFunc(namespace string) func(interface{}) bool {
 			f.logger.Warn("Found invalid Lease holder identify ", holder)
 			return false
 		}
-		if p := f.getProcessor(l.Name); p != nil && p.is(holder) {
+		if p := f.fwd.getProcessor(l.Name); p != nil && p.is(holder) {
 			return false
 		}
 
@@ -180,7 +199,7 @@ func (f *Forwarder) filterFunc(namespace string) func(interface{}) bool {
 	}
 }
 
-func (f *Forwarder) leaseUpdated(obj interface{}) {
+func (f *leaseTracker) leaseUpdated(obj interface{}) {
 	l := obj.(*coordinationv1.Lease)
 	ns, n := l.Namespace, l.Name
 	ctx := context.Background()
@@ -194,7 +213,10 @@ func (f *Forwarder) leaseUpdated(obj interface{}) {
 
 	// Close existing connection if there's one. The target address won't
 	// be the same as the IP has changed.
-	f.shutdown(f.getProcessor(n))
+	if p := f.fwd.getProcessor(n); p != nil {
+		p.shutdown()
+	}
+
 	holder := *l.Spec.HolderIdentity
 	// The map should have the IP because of the operations in the filter when queueing.
 	ip, ok := f.id2ip[holder]
@@ -204,13 +226,23 @@ func (f *Forwarder) leaseUpdated(obj interface{}) {
 		f.logger.Warn("IP not found in cached map for ", holder)
 		return
 	}
-	f.setProcessor(n, f.createProcessor(ns, n, ip, holder))
 
 	if ip != f.selfIP {
+		f.fwd.setProcessor(n, newForwardProcessor(f.logger.With(zap.String("bucket", n)), n, holder,
+			fmt.Sprintf("ws://%s:%d", ip, autoscalerPort),
+			fmt.Sprintf("ws://%s.%s.%s", n, ns, svcURLSuffix)))
+
 		// Skip creating/updating Service and Endpoints if not the leader.
 		return
 	}
 
+	f.fwd.setProcessor(n, &localProcessor{
+		bkt:    n,
+		holder: holder,
+		logger: f.logger.With(zap.String("bucket", n)),
+		accept: f.accept,
+	})
+
 	if err := f.createService(ctx, ns, n); err != nil {
 		f.logger.Fatalf("Failed to create Service for Lease %s/%s: %v", ns, n, err)
 		return
@@ -224,7 +256,7 @@ func (f *Forwarder) leaseUpdated(obj interface{}) {
 	f.logger.Infof("Created Endpoints for Lease %s/%s", ns, n)
 }
 
-func (f *Forwarder) createService(ctx context.Context, ns, n string) error {
+func (f *leaseTracker) createService(ctx context.Context, ns, n string) error {
 	var lastErr error
 	if err := wait.PollImmediate(retryInterval, retryTimeout, func() (bool, error) {
 		_, lastErr = f.kc.CoreV1().Services(ns).Create(ctx, &corev1.Service{
@@ -257,7 +289,7 @@ func (f *Forwarder) createService(ctx context.Context, ns, n string) error {
 // createOrUpdateEndpoints creates an Endpoints object with the given namespace and
 // name, and the Forwarder.selfIP. If the Endpoints object already
 // exists, it will update the Endpoints with the Forwarder.selfIP.
-func (f *Forwarder) createOrUpdateEndpoints(ctx context.Context, ns, n string) error {
+func (f *leaseTracker) createOrUpdateEndpoints(ctx context.Context, ns, n string) error {
 	wantSubsets := []corev1.EndpointSubset{{
 		Addresses: []corev1.EndpointAddress{{
 			IP: f.selfIP,
@@ -334,21 +366,6 @@ func (f *Forwarder) setProcessor(bkt string, p bucketProcessor) {
 	f.processors[bkt] = p
 }
 
-func (f *Forwarder) createProcessor(ns, bkt, ip, holder string) bucketProcessor {
-	if ip == f.selfIP {
-		return &localProcessor{
-			bkt:    bkt,
-			holder: holder,
-			logger: f.logger.With(zap.String("bucket", bkt)),
-			accept: f.accept,
-		}
-	}
-
-	return newForwardProcessor(f.logger.With(zap.String("bucket", bkt)), bkt, holder,
-		fmt.Sprintf("ws://%s:%d", ip, autoscalerPort),
-		fmt.Sprintf("ws://%s.%s.%s", bkt, ns, svcURLSuffix))
-}
-
 // Process enqueues the given Stat for processing asynchronously.
 // It calls Forwarder.accept if the pod where this Forwarder is running is the owner
 // of the given StatMessage. Otherwise it forwards the given StatMessage to the right
@@ -403,12 +420,6 @@ func (f *Forwarder) maybeRetry(logger *zap.SugaredLogger, s stat) {
 	}()
 }
 
-func (f *Forwarder) shutdown(p bucketProcessor) {
-	if p != nil {
-		p.shutdown()
-	}
-}
-
 // Cancel is the function to call when terminating a Forwarder.
 func (f *Forwarder) Cancel() {
 	// Tell process go-runtine to stop.
@@ -417,7 +428,9 @@ func (f *Forwarder) Cancel() {
 	f.processorsLock.RLock()
 	defer f.processorsLock.RUnlock()
 	for _, p := range f.processors {
-		f.shutdown(p)
+		if p != nil {
+			p.shutdown()
+		}
 	}
 
 	f.processingWg.Wait()
diff --git a/pkg/autoscaler/statforwarder/processor.go b/pkg/autoscaler/statforwarder/processor.go
index 3f2ab18b4..358735d7e 100644
--- a/pkg/autoscaler/statforwarder/processor.go
+++ b/pkg/autoscaler/statforwarder/processor.go
@@ -74,8 +74,13 @@ type remoteProcessor struct {
 	// The name of the bucket
 	bkt string
 	// holder is the HolderIdentity of a Lease for a bucket.
-	holder   string
-	svcDNS   string
+	holder string
+
+	// addrs contains the list of addresses for the remote to try in the order to try them.
+	// The first address to establish a successful connection will be used until things reset
+	// at which point the order is re-evaluated.
+	addrs []string
+
 	connLock sync.RWMutex
 	// conn is the WebSocket connection to the holder pod.
 	conn *websocket.ManagedConnection
@@ -83,22 +88,12 @@ type remoteProcessor struct {
 
 var _ bucketProcessor = (*remoteProcessor)(nil)
 
-func newForwardProcessor(logger *zap.SugaredLogger, bkt, holder, podDNS, svcDNS string) *remoteProcessor {
-	// First try to connect via Pod IP address synchronously. If the connection can
-	// not be established within `establishTimeout`, we assume the pods can not be
-	// accessed by IP address. Then try to connect via Pod IP address synchronously.
-	logger.Info("Connecting to Autoscaler bucket at ", podDNS)
-	c, err := websocket.NewDurableSendingConnectionGuaranteed(podDNS, establishTimeout, logger)
-	if err != nil {
-		logger.Info("Autoscaler pods can't be accessed by IP address. Connecting to Autoscaler bucket at ", svcDNS)
-		c, _ = websocket.NewDurableSendingConnectionGuaranteed(svcDNS, establishTimeout, logger)
-	}
+func newForwardProcessor(logger *zap.SugaredLogger, bkt, holder string, addrs ...string) *remoteProcessor {
 	return &remoteProcessor{
 		logger: logger,
 		bkt:    bkt,
 		holder: holder,
-		conn:   c,
-		svcDNS: svcDNS,
+		addrs:  addrs,
 	}
 }
 
@@ -130,11 +125,17 @@ func (p *remoteProcessor) process(sm asmetrics.StatMessage) error {
 
 	c := p.getConn()
 	if c == nil {
-		c, err = websocket.NewDurableSendingConnectionGuaranteed(p.svcDNS, establishTimeout, p.logger)
+		for _, addr := range p.addrs {
+			c, err = websocket.NewDurableSendingConnectionGuaranteed(addr, establishTimeout, p.logger)
+			if err != nil {
+				continue
+			}
+			p.setConn(c)
+			break
+		}
 		if err != nil {
 			return err
 		}
-		p.setConn(c)
 	}
 
 	return c.SendRaw(gorillawebsocket.BinaryMessage, b)
diff --git a/pkg/reconciler/route/traffic/rollout.go b/pkg/reconciler/route/traffic/rollout.go
index c9ac44c38..333b34be9 100644
--- a/pkg/reconciler/route/traffic/rollout.go
+++ b/pkg/reconciler/route/traffic/rollout.go
@@ -58,10 +58,13 @@ type ConfigurationRollout struct {
 	// Note: that it is not 100% of the route traffic, in more complex cases.
 	Revisions []RevisionRollout `json:"revisions,omitempty"`
 
-	// Deadline is the Unix timestamp by when (+/- reconcile precision)
-	// the Rollout shoud be complete.
-	Deadline int `json:"deadline,omitempty"`
+	// StepParams describes rollout params for the configuration.
+	StepParams RolloutParams `json:"stepParams"`
+}
 
+// RolloutParams contains the timing and sizing parameters for the
+// ConfigurationRollout.
+type RolloutParams struct {
 	// StartTime is the Unix timestamp by when (+/- reconcile precision)
 	// the Rollout has started.
 	// This is required to compute step time and deadline.
@@ -128,10 +131,10 @@ const durationSecs = 120.0
 func (cur *Rollout) ObserveReady(nowTS int) {
 	for i := range cur.Configurations {
 		c := &cur.Configurations[i]
-		if c.StepDuration == 0 && c.StartTime > 0 {
-			// In really ceil(nowTS-c.StartTime) should always give 1s, but
+		if c.StepParams.StepDuration == 0 && c.StepParams.StartTime > 0 {
+			// In really ceil(nowTS-params.StartTime) should always give 1s, but
 			// given possible time drift, we'll ensure that at least 1s is returned.
-			minStepSec := math.Max(1, math.Ceil(time.Duration(nowTS-c.StartTime).Seconds()))
+			minStepSec := math.Max(1, math.Ceil(time.Duration(nowTS-c.StepParams.StartTime).Seconds()))
 			c.computeProperties(float64(nowTS), minStepSec, durationSecs)
 		}
 	}
@@ -248,12 +251,12 @@ func adjustPercentage(goal int, cr *ConfigurationRollout) {
 func stepRevisions(goal *ConfigurationRollout, nowTS int) {
 	// Not yet ready to adjust the steps or we're done
 	// (shouldn't really be here, but better be defensive).
-	if nowTS < goal.NextStepTime || len(goal.Revisions) < 2 {
+	if nowTS < goal.StepParams.NextStepTime || len(goal.Revisions) < 2 {
 		return
 	}
 
 	revLen := len(goal.Revisions)
-	remaining := goal.StepSize
+	remaining := goal.StepParams.StepSize
 	writePos := revLen - 1
 	// readPos is guaranteed to be >= 0, due to the check above.
 	readPos := revLen - 2
@@ -283,7 +286,7 @@ func stepRevisions(goal *ConfigurationRollout, nowTS int) {
 	// Copy the last one to the write pos
 	goal.Revisions[writePos] = goal.Revisions[revLen-1]
 
-	goal.Revisions[writePos].Percent += goal.StepSize
+	goal.Revisions[writePos].Percent += goal.StepParams.StepSize
 	// This can happen if step is now larger than total allocation, see the
 	// note above.
 	// E.g. with example above R2 = 20, and ro we have to cap it at 15.
@@ -294,11 +297,10 @@ func stepRevisions(goal *ConfigurationRollout, nowTS int) {
 	goal.Revisions = goal.Revisions[:writePos+1]
 	// Also set the next time.
 	if len(goal.Revisions) > 1 {
-		goal.NextStepTime = nowTS + goal.StepDuration
+		goal.StepParams.NextStepTime = nowTS + goal.StepParams.StepDuration
 	} else {
-		// This is the last step, we're done!
-		goal.NextStepTime = 0
-		goal.StartTime = 0
+		// This is the last step, we're done! Clear the params out.
+		goal.StepParams = RolloutParams{}
 	}
 }
 
@@ -334,14 +336,10 @@ func stepConfig(goal, prev *ConfigurationRollout, nowTS int) *ConfigurationRollo
 
 			// Copy various rollout stats from the previous when no new revision
 			// has been created.
-			ret.StartTime = prev.StartTime
-			ret.Deadline = prev.Deadline
-			ret.NextStepTime = prev.NextStepTime
-			ret.StepDuration = prev.StepDuration
-			ret.StepSize = prev.StepSize
+			ret.StepParams = prev.StepParams
 			// We might end up here before `ObserveReady` is called.
 			// In that case don't step individual revisions just yet.
-			if ret.StepSize > 0 {
+			if ret.StepParams.StepSize > 0 {
 				// adjustPercentage above would've already accounted if target for the
 				// whole Configuration changed up or down. So here we should just redistribute
 				// the existing values.
@@ -354,7 +352,7 @@ func stepConfig(goal, prev *ConfigurationRollout, nowTS int) *ConfigurationRollo
 	// Otherwise we start a rollout, which means we need to stamp the starttime,
 	// the rest of the fields will remain unset and `ObserveReady` will
 	// compute them when the ingress becomes ready.
-	ret.StartTime = nowTS
+	ret.StepParams.StartTime = nowTS
 
 	// Go backwards and find first revision with traffic assignment > 0.
 	// Reduce it by one, so we can give that 1% to the new revision.
@@ -417,9 +415,9 @@ func (cur *ConfigurationRollout) computeProperties(nowTS, minStepSec, durationSe
 	// than slightly shorter.
 	stepDuration := math.Ceil(durationSecs / numSteps)
 
-	cur.StepDuration = int(stepDuration)
-	cur.StepSize = int(stepSize)
-	cur.NextStepTime = int(nowTS + stepDuration*float64(time.Second))
+	cur.StepParams.StepDuration = int(stepDuration)
+	cur.StepParams.StepSize = int(stepSize)
+	cur.StepParams.NextStepTime = int(nowTS + stepDuration*float64(time.Second))
 }
 
 // sortRollout sorts the rollout based on tag so it's consistent
diff --git a/vendor/knative.dev/pkg/leaderelection/context.go b/vendor/knative.dev/pkg/leaderelection/context.go
index 17a7a1b1f..d30ecd462 100644
--- a/vendor/knative.dev/pkg/leaderelection/context.go
+++ b/vendor/knative.dev/pkg/leaderelection/context.go
@@ -41,7 +41,7 @@ func WithDynamicLeaderElectorBuilder(ctx context.Context, kc kubernetes.Interfac
 	b, _, err := NewStatefulSetBucketAndSet(int(cc.Buckets))
 	if err == nil {
 		logger.Info("Running with StatefulSet leader election")
-		return withStatefulSetElectorBuilder(ctx, cc, b)
+		return WithStatefulSetElectorBuilder(ctx, cc, b)
 	}
 	logger.Info("Running with Standard leader election")
 	return WithStandardLeaderElectorBuilder(ctx, kc, cc)
@@ -57,10 +57,10 @@ func WithStandardLeaderElectorBuilder(ctx context.Context, kc kubernetes.Interfa
 	})
 }
 
-// withStatefulSetElectorBuilder infuses a context with the ability to build
+// WithStatefulSetElectorBuilder infuses a context with the ability to build
 // Electors which are assigned leadership based on the StatefulSet ordinal from
 // the provided component configuration.
-func withStatefulSetElectorBuilder(ctx context.Context, cc ComponentConfig, bkt reconciler.Bucket) context.Context {
+func WithStatefulSetElectorBuilder(ctx context.Context, cc ComponentConfig, bkt reconciler.Bucket) context.Context {
 	return context.WithValue(ctx, builderKey{}, &statefulSetBuilder{
 		lec: cc,
 		bkt: bkt,
diff --git a/vendor/knative.dev/pkg/webhook/helper.go b/vendor/knative.dev/pkg/webhook/helper.go
new file mode 100644
index 000000000..cbcbb58c7
--- /dev/null
+++ b/vendor/knative.dev/pkg/webhook/helper.go
@@ -0,0 +1,64 @@
+/*
+Copyright 2020 The Knative Authors
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package webhook
+
+import (
+	"strings"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+// EnsureLabelSelectorExpressions merges the current label selector's MatchExpressions
+// with the ones wanted.
+// It keeps all non-knative keys intact, removes all knative-keys no longer wanted and
+// adds all knative-keys not yet there.
+func EnsureLabelSelectorExpressions(
+	current *metav1.LabelSelector,
+	want *metav1.LabelSelector) *metav1.LabelSelector {
+
+	if current == nil {
+		return want
+	}
+
+	if len(current.MatchExpressions) == 0 {
+		return want
+	}
+
+	var wantExpressions []metav1.LabelSelectorRequirement
+	if want != nil {
+		wantExpressions = want.MatchExpressions
+	}
+
+	return &metav1.LabelSelector{
+		MatchExpressions: ensureLabelSelectorRequirements(
+			current.MatchExpressions, wantExpressions),
+	}
+}
+
+func ensureLabelSelectorRequirements(
+	current []metav1.LabelSelectorRequirement,
+	want []metav1.LabelSelectorRequirement) []metav1.LabelSelectorRequirement {
+
+	nonKnative := make([]metav1.LabelSelectorRequirement, 0, len(current))
+	for _, r := range current {
+		if !strings.Contains(r.Key, "knative.dev") {
+			nonKnative = append(nonKnative, r)
+		}
+	}
+
+	return append(want, nonKnative...)
+}
diff --git a/vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/defaulting.go b/vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/defaulting.go
index 69d0e9ae8..acdc6f33d 100644
--- a/vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/defaulting.go
+++ b/vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/defaulting.go
@@ -171,41 +171,42 @@ func (ac *reconciler) reconcileMutatingWebhook(ctx context.Context, caCert []byt
 		return fmt.Errorf("error retrieving webhook: %w", err)
 	}
 
-	webhook := configuredWebhook.DeepCopy()
+	current := configuredWebhook.DeepCopy()
 
 	// Clear out any previous (bad) OwnerReferences.
 	// See: https://github.com/knative/serving/issues/5845
-	webhook.OwnerReferences = nil
+	current.OwnerReferences = nil
 
-	for i, wh := range webhook.Webhooks {
-		if wh.Name != webhook.Name {
+	for i, wh := range current.Webhooks {
+		if wh.Name != current.Name {
 			continue
 		}
-		webhook.Webhooks[i].Rules = rules
-		webhook.Webhooks[i].NamespaceSelector = &metav1.LabelSelector{
-			MatchExpressions: []metav1.LabelSelectorRequirement{{
-				Key:      "webhooks.knative.dev/exclude",
-				Operator: metav1.LabelSelectorOpDoesNotExist,
-			}, {
-				// "control-plane" is added to support Azure's AKS, otherwise the controllers fight.
-				// See knative/pkg#1590 for details.
-				Key:      "control-plane",
-				Operator: metav1.LabelSelectorOpDoesNotExist,
-			}},
-		}
-		webhook.Webhooks[i].ClientConfig.CABundle = caCert
-		if webhook.Webhooks[i].ClientConfig.Service == nil {
+
+		cur := &current.Webhooks[i]
+		cur.Rules = rules
+
+		cur.NamespaceSelector = webhook.EnsureLabelSelectorExpressions(
+			cur.NamespaceSelector,
+			&metav1.LabelSelector{
+				MatchExpressions: []metav1.LabelSelectorRequirement{{
+					Key:      "webhooks.knative.dev/exclude",
+					Operator: metav1.LabelSelectorOpDoesNotExist,
+				}},
+			})
+
+		cur.ClientConfig.CABundle = caCert
+		if cur.ClientConfig.Service == nil {
 			return fmt.Errorf("missing service reference for webhook: %s", wh.Name)
 		}
-		webhook.Webhooks[i].ClientConfig.Service.Path = ptr.String(ac.Path())
+		cur.ClientConfig.Service.Path = ptr.String(ac.Path())
 	}
 
-	if ok, err := kmp.SafeEqual(configuredWebhook, webhook); err != nil {
+	if ok, err := kmp.SafeEqual(configuredWebhook, current); err != nil {
 		return fmt.Errorf("error diffing webhooks: %w", err)
 	} else if !ok {
 		logger.Info("Updating webhook")
 		mwhclient := ac.client.AdmissionregistrationV1().MutatingWebhookConfigurations()
-		if _, err := mwhclient.Update(ctx, webhook, metav1.UpdateOptions{}); err != nil {
+		if _, err := mwhclient.Update(ctx, current, metav1.UpdateOptions{}); err != nil {
 			return fmt.Errorf("failed to update webhook: %w", err)
 		}
 	} else {
diff --git a/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/reconcile_config.go b/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/reconcile_config.go
index 281a12e43..3abe55e16 100644
--- a/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/reconcile_config.go
+++ b/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/reconcile_config.go
@@ -134,41 +134,41 @@ func (ac *reconciler) reconcileValidatingWebhook(ctx context.Context, caCert []b
 		return fmt.Errorf("error retrieving webhook: %w", err)
 	}
 
-	webhook := configuredWebhook.DeepCopy()
+	current := configuredWebhook.DeepCopy()
 
 	// Clear out any previous (bad) OwnerReferences.
 	// See: https://github.com/knative/serving/issues/5845
-	webhook.OwnerReferences = nil
+	current.OwnerReferences = nil
 
-	for i, wh := range webhook.Webhooks {
-		if wh.Name != webhook.Name {
+	for i, wh := range current.Webhooks {
+		if wh.Name != current.Name {
 			continue
 		}
-		webhook.Webhooks[i].Rules = rules
-		webhook.Webhooks[i].NamespaceSelector = &metav1.LabelSelector{
-			MatchExpressions: []metav1.LabelSelectorRequirement{{
-				Key:      "webhooks.knative.dev/exclude",
-				Operator: metav1.LabelSelectorOpDoesNotExist,
-			}, {
-				// "control-plane" is added to support Azure's AKS, otherwise the controllers fight.
-				// See knative/pkg#1590 for details.
-				Key:      "control-plane",
-				Operator: metav1.LabelSelectorOpDoesNotExist,
-			}},
-		}
-		webhook.Webhooks[i].ClientConfig.CABundle = caCert
-		if webhook.Webhooks[i].ClientConfig.Service == nil {
+		cur := &current.Webhooks[i]
+		cur.Rules = rules
+
+		cur.NamespaceSelector = webhook.EnsureLabelSelectorExpressions(
+			cur.NamespaceSelector,
+			&metav1.LabelSelector{
+				MatchExpressions: []metav1.LabelSelectorRequirement{{
+					Key:      "webhooks.knative.dev/exclude",
+					Operator: metav1.LabelSelectorOpDoesNotExist,
+				}},
+			})
+
+		cur.ClientConfig.CABundle = caCert
+		if cur.ClientConfig.Service == nil {
 			return fmt.Errorf("missing service reference for webhook: %s", wh.Name)
 		}
-		webhook.Webhooks[i].ClientConfig.Service.Path = ptr.String(ac.Path())
+		cur.ClientConfig.Service.Path = ptr.String(ac.Path())
 	}
 
-	if ok, err := kmp.SafeEqual(configuredWebhook, webhook); err != nil {
+	if ok, err := kmp.SafeEqual(configuredWebhook, current); err != nil {
 		return fmt.Errorf("error diffing webhooks: %w", err)
 	} else if !ok {
 		logger.Info("Updating webhook")
 		vwhclient := ac.client.AdmissionregistrationV1().ValidatingWebhookConfigurations()
-		if _, err := vwhclient.Update(ctx, webhook, metav1.UpdateOptions{}); err != nil {
+		if _, err := vwhclient.Update(ctx, current, metav1.UpdateOptions{}); err != nil {
 			return fmt.Errorf("failed to update webhook: %w", err)
 		}
 	} else {
diff --git a/vendor/modules.txt b/vendor/modules.txt
index fc16696b4..12fff9c67 100644
--- a/vendor/modules.txt
+++ b/vendor/modules.txt
@@ -978,7 +978,7 @@ knative.dev/caching/pkg/client/listers/caching/v1alpha1
 # knative.dev/hack v0.0.0-20201201234937-fddbf732e450
 ## explicit
 knative.dev/hack
-# knative.dev/networking v0.0.0-20201208014140-4d1d53eb0626
+# knative.dev/networking v0.0.0-20201209014241-22a7e22628fc
 ## explicit
 knative.dev/networking/config
 knative.dev/networking/pkg
@@ -1012,7 +1012,7 @@ knative.dev/networking/test/conformance/ingress
 knative.dev/networking/test/defaultsystem
 knative.dev/networking/test/test_images/grpc-ping/proto
 knative.dev/networking/test/types
-# knative.dev/pkg v0.0.0-20201209011241-b1ac587fdd80
+# knative.dev/pkg v0.0.0-20201209172941-9b9407a76df1
 ## explicit
 knative.dev/pkg/apiextensions/storageversion
 knative.dev/pkg/apiextensions/storageversion/cmd/migrate


Test patch:
diff --git a/.github/workflows/knative-go-test.yaml b/.github/workflows/knative-go-test.yaml
index 9a77276c1..965df1e74 100644
--- a/.github/workflows/knative-go-test.yaml
+++ b/.github/workflows/knative-go-test.yaml
@@ -20,10 +20,10 @@ name: Test
 on:
 
   push:
-    branches: [ 'master' ]
+    branches: [ 'main', 'master' ]
 
   pull_request:
-    branches: [ 'master', 'release-*' ]
+    branches: [ 'main', 'master', 'release-*' ]
 
 jobs:
 
diff --git a/pkg/autoscaler/statforwarder/processor_test.go b/pkg/autoscaler/statforwarder/processor_test.go
index 211e3f453..8fe166116 100644
--- a/pkg/autoscaler/statforwarder/processor_test.go
+++ b/pkg/autoscaler/statforwarder/processor_test.go
@@ -79,7 +79,7 @@ func TestProcessorForwardingViaSvcRetry(t *testing.T) {
 	}
 
 	// Change to a working URL
-	p.svcDNS = "ws" + strings.TrimPrefix(s.URL, "http")
+	p.addrs = []string{"ws" + strings.TrimPrefix(s.URL, "http")}
 	p.process(stat1)
 
 	select {
diff --git a/pkg/reconciler/route/resources/ingress_test.go b/pkg/reconciler/route/resources/ingress_test.go
index ac1339e87..03a9db11c 100644
--- a/pkg/reconciler/route/resources/ingress_test.go
+++ b/pkg/reconciler/route/resources/ingress_test.go
@@ -141,7 +141,7 @@ func TestMakeIngressWithRollout(t *testing.T) {
 		Annotations: map[string]string{
 			// Make sure to get passdownIngressClass instead of ingressClass
 			networking.IngressClassAnnotationKey: passdownIngressClass,
-			networking.RolloutAnnotationKey:      `{"configurations":[{"configurationName":"valhalla","percent":100,"revisions":[{"revisionName":"valhalla-01982","percent":100}]},{"configurationName":"thor","tag":"tagged","percent":100,"revisions":[{"revisionName":"thor-02020","percent":100}]}]}`,
+			networking.RolloutAnnotationKey:      `{"configurations":[{"configurationName":"valhalla","percent":100,"revisions":[{"revisionName":"valhalla-01982","percent":100}],"stepParams":{}},{"configurationName":"thor","tag":"tagged","percent":100,"revisions":[{"revisionName":"thor-02020","percent":100}],"stepParams":{}}]}`,
 			"test-annotation":                    "bar",
 		},
 		OwnerReferences: []metav1.OwnerReference{*kmeta.NewControllerRef(r)},
diff --git a/pkg/reconciler/route/table_test.go b/pkg/reconciler/route/table_test.go
index ecc4b304b..b8ca78975 100644
--- a/pkg/reconciler/route/table_test.go
+++ b/pkg/reconciler/route/table_test.go
@@ -436,11 +436,11 @@ func TestReconcile(t *testing.T) {
 				}}, fakeCurTime.Add(-3*time.Second),
 					func(r *traffic.Rollout) {
 						// Step duration is 3s (now - (now-3s)).
-						r.Configurations[0].StepDuration = 3
+						r.Configurations[0].StepParams.StepDuration = 3
 						// 120 / 3 = 40 steps. floor(100/40) = 2.
-						r.Configurations[0].StepSize = 2
+						r.Configurations[0].StepParams.StepSize = 2
 						// StepDuration is 3, and so next step is `now` + 3.
-						r.Configurations[0].NextStepTime = int(fakeCurTime.Add(3 * time.Second).UnixNano())
+						r.Configurations[0].StepParams.NextStepTime = int(fakeCurTime.Add(3 * time.Second).UnixNano())
 					},
 				)),
 		}, {
@@ -2694,9 +2694,11 @@ func simpleRollout(cfg string, revs []traffic.RevisionRollout,
 		r := &traffic.Rollout{
 			Configurations: []traffic.ConfigurationRollout{{
 				ConfigurationName: cfg,
-				StartTime:         int(now.UnixNano()),
-				Percent:           100,
-				Revisions:         revs,
+				StepParams: traffic.RolloutParams{
+					StartTime: int(now.UnixNano()),
+				},
+				Percent:   100,
+				Revisions: revs,
 			}},
 		}
 		for _, ro := range ros {
diff --git a/pkg/reconciler/route/traffic/rollout_test.go b/pkg/reconciler/route/traffic/rollout_test.go
index 8927d5143..d0df53810 100644
--- a/pkg/reconciler/route/traffic/rollout_test.go
+++ b/pkg/reconciler/route/traffic/rollout_test.go
@@ -78,11 +78,12 @@ func TestStep(t *testing.T) {
 					RevisionName: "let-it-bleed",
 					Percent:      100,
 				}},
-				Deadline:     2006, // <- Those should be kept.
-				NextStepTime: 2009,
-				StepDuration: 2020,
-				StartTime:    2004,
-				StepSize:     14,
+				StepParams: RolloutParams{ // <- Those should be kept.
+					NextStepTime: 2009,
+					StepDuration: 2020,
+					StartTime:    2004,
+					StepSize:     14,
+				},
 			}},
 		},
 		prev: &Rollout{
@@ -115,20 +116,24 @@ func TestStep(t *testing.T) {
 					RevisionName: "let-it-bleed",
 					Percent:      100,
 				}},
-				NextStepTime: 2009,
-				StepDuration: 2020,
-				StartTime:    2004,
-				StepSize:     14,
+				StepParams: RolloutParams{
+					NextStepTime: 2009,
+					StepDuration: 2020,
+					StartTime:    2004,
+					StepSize:     14,
+				},
 			}},
 		},
 		prev: &Rollout{
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           100,
-				NextStepTime:      2019,
-				StepDuration:      5,
-				StartTime:         2004,
-				StepSize:          3,
+				StepParams: RolloutParams{
+					NextStepTime: 2019,
+					StepDuration: 5,
+					StartTime:    2004,
+					StepSize:     3,
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "their-satanic-majesties-request",
 					Percent:      95,
@@ -149,10 +154,12 @@ func TestStep(t *testing.T) {
 					RevisionName: "let-it-bleed",
 					Percent:      8, // +3
 				}},
-				NextStepTime: 2020 + 5, // now + duration.
-				StepDuration: 5,
-				StartTime:    2004,
-				StepSize:     3,
+				StepParams: RolloutParams{
+					NextStepTime: 2020 + 5, // now + duration.
+					StepDuration: 5,
+					StartTime:    2004,
+					StepSize:     3,
+				},
 			}},
 		},
 	}, {
@@ -165,21 +172,24 @@ func TestStep(t *testing.T) {
 					RevisionName: "let-it-bleed",
 					Percent:      100,
 				}},
-				Deadline:     2006,
-				NextStepTime: 2009,
-				StepDuration: 2020,
-				StartTime:    2004,
-				StepSize:     14,
+				StepParams: RolloutParams{
+					NextStepTime: 2009,
+					StepDuration: 2020,
+					StartTime:    2004,
+					StepSize:     14,
+				},
 			}},
 		},
 		prev: &Rollout{
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           100,
-				NextStepTime:      2019, // <- Those should be reset and/or updated.
-				StartTime:         2004,
-				StepDuration:      5,
-				StepSize:          0,
+				StepParams: RolloutParams{
+					NextStepTime: 2019, // <- Those should be reset and/or updated.
+					StartTime:    2004,
+					StepDuration: 5,
+					StepSize:     0,
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "their-satanic-majesties-request",
 					Percent:      95,
@@ -200,10 +210,12 @@ func TestStep(t *testing.T) {
 					RevisionName: "let-it-bleed",
 					Percent:      5,
 				}},
-				NextStepTime: 2019,
-				StartTime:    2004,
-				StepDuration: 5,
-				StepSize:     0,
+				StepParams: RolloutParams{
+					NextStepTime: 2019,
+					StartTime:    2004,
+					StepDuration: 5,
+					StepSize:     0,
+				},
 			}},
 		},
 	}, {
@@ -288,17 +300,20 @@ func TestStep(t *testing.T) {
 					RevisionName: "goat-head-soup",
 					Percent:      100,
 				}},
-				Deadline:     1982, // <- Those should be thrown out.
-				NextStepTime: 1984,
-				StepDuration: 1988,
-				StepSize:     11,
+				StepParams: RolloutParams{ // <- Those should be thrown out.
+					NextStepTime: 1984,
+					StepDuration: 1988,
+					StepSize:     11,
+				},
 			}},
 		},
 		want: &Rollout{
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           100,
-				StartTime:         now,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      99,
@@ -337,7 +352,9 @@ func TestStep(t *testing.T) {
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           33,
-				StartTime:         now,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      2,
@@ -378,8 +395,10 @@ func TestStep(t *testing.T) {
 		want: &Rollout{
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
-				StartTime:         now,
-				Percent:           75,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
+				Percent: 75,
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      11,
@@ -418,7 +437,9 @@ func TestStep(t *testing.T) {
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "brian",
 				Percent:           70,
-				StartTime:         now,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "exile-on-main-st",
 					Percent:      69,
@@ -444,7 +465,9 @@ func TestStep(t *testing.T) {
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           100,
-				StartTime:         now - 1982, // A rollout in progress, this would be set.
+				StepParams: RolloutParams{
+					StartTime: now - 1982, // A rollout in progress, this would be set.
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      95,
@@ -458,7 +481,9 @@ func TestStep(t *testing.T) {
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           100,
-				StartTime:         now,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      95,
@@ -487,7 +512,9 @@ func TestStep(t *testing.T) {
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
 				Percent:           100,
-				StartTime:         now - 1984, // A rollout in progress, this would be set.
+				StepParams: RolloutParams{
+					StartTime: now - 1984, // A rollout in progress, this would be set.
+				},
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      99,
@@ -500,8 +527,10 @@ func TestStep(t *testing.T) {
 		want: &Rollout{
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "mick",
-				StartTime:         now,
-				Percent:           100,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
+				Percent: 100,
 				Revisions: []RevisionRollout{{
 					RevisionName: "goat-head-soup",
 					Percent:      99,
@@ -670,7 +699,9 @@ func TestStep(t *testing.T) {
 			Configurations: []ConfigurationRollout{{
 				ConfigurationName: "keith",
 				Percent:           99,
-				StartTime:         now,
+				StepParams: RolloutParams{
+					StartTime: now,
+				},
 				Revisions: []RevisionRollout{{ // <-- note this one actually rolls.
 					RevisionName: "can't-get-no-satisfaction",
 					Percent:      98,
@@ -803,22 +834,30 @@ func TestObserveReady(t *testing.T) {
 		Configurations: []ConfigurationRollout{{
 			Percent:           100,
 			ConfigurationName: "has-step",
-			StepDuration:      11,
+			StepParams: RolloutParams{
+				StepDuration: 11,
+			},
 		}, {
 			ConfigurationName: "no-step-no-begin",
 			Percent:           100,
 		}, {
 			ConfigurationName: "step-begin < 1s",
-			StartTime:         200620092020,
-			Percent:           100,
+			StepParams: RolloutParams{
+				StartTime: 200620092020,
+			},
+			Percent: 100,
 		}, {
 			ConfigurationName: "step-begin > 1s",
-			StartTime:         oldenDays,
-			Percent:           100,
+			StepParams: RolloutParams{
+				StartTime: oldenDays,
+			},
+			Percent: 100,
 		}, {
 			ConfigurationName: "Percent not 100%",
-			StartTime:         oldenDays,
-			Percent:           50,
+			StepParams: RolloutParams{
+				StartTime: oldenDays,
+			},
+			Percent: 50,
 		}},
 	}
 
@@ -826,31 +865,39 @@ func TestObserveReady(t *testing.T) {
 		Configurations: []ConfigurationRollout{{
 			Percent:           100,
 			ConfigurationName: "has-step",
-			StepDuration:      11,
+			StepParams: RolloutParams{
+				StepDuration: 11,
+			},
 		}, {
 			ConfigurationName: "no-step-no-begin",
 			Percent:           100,
 		}, {
 			ConfigurationName: "step-begin < 1s",
 			Percent:           100,
-			StartTime:         200620092020,
-			StepDuration:      2,
-			StepSize:          1,
-			NextStepTime:      now + int(2*time.Second),
+			StepParams: RolloutParams{
+				StartTime:    200620092020,
+				StepDuration: 2,
+				StepSize:     1,
+				NextStepTime: now + int(2*time.Second),
+			},
 		}, {
 			ConfigurationName: "step-begin > 1s",
 			Percent:           100,
-			StartTime:         oldenDays,
-			StepDuration:      3,
-			StepSize:          100 / 40,
-			NextStepTime:      now + 3*int(time.Second),
+			StepParams: RolloutParams{
+				StartTime:    oldenDays,
+				StepDuration: 3,
+				StepSize:     100 / 40,
+				NextStepTime: now + 3*int(time.Second),
+			},
 		}, {
 			ConfigurationName: "Percent not 100%",
 			Percent:           50,
-			StartTime:         oldenDays,
-			StepDuration:      3,
-			StepSize:          50 / 40,
-			NextStepTime:      now + 3*int(time.Second),
+			StepParams: RolloutParams{
+				StartTime:    oldenDays,
+				StepDuration: 3,
+				StepSize:     50 / 40,
+				NextStepTime: now + 3*int(time.Second),
+			},
 		}},
 	}
 
@@ -1110,10 +1157,11 @@ func TestJSONRoundtrip(t *testing.T) {
 				RevisionName: "roy",
 				Percent:      100,
 			}},
-			StartTime:    1955,
-			Deadline:     2006,
-			NextStepTime: 1988,
-			StepDuration: 1984,
+			StepParams: RolloutParams{
+				StartTime:    1955,
+				NextStepTime: 1988,
+				StepDuration: 1984,
+			},
 		}, {
 			ConfigurationName: "no",
 			Percent:           0,
@@ -1156,8 +1204,10 @@ func TestStepRevisions(t *testing.T) {
 		name: "noop (1): too soon",
 		now:  1982,
 		cfg: &ConfigurationRollout{
-			NextStepTime: 1984,
-			StepSize:     10,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepSize:     10,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 10,
 			}, {
@@ -1167,8 +1217,10 @@ func TestStepRevisions(t *testing.T) {
 			}},
 		},
 		want: &ConfigurationRollout{
-			NextStepTime: 1984,
-			StepSize:     10,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepSize:     10,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 10,
 			}, {
@@ -1181,17 +1233,21 @@ func TestStepRevisions(t *testing.T) {
 		name: "noop (2): done",
 		now:  1982,
 		cfg: &ConfigurationRollout{
-			NextStepTime: 1984,
-			StepDuration: 10,
-			StepSize:     10,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepDuration: 10,
+				StepSize:     10,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 10,
 			}},
 		},
 		want: &ConfigurationRollout{
-			NextStepTime: 1984,
-			StepDuration: 10,
-			StepSize:     10,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepDuration: 10,
+				StepSize:     10,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 10,
 			}},
@@ -1200,10 +1256,12 @@ func TestStepRevisions(t *testing.T) {
 		name: "step last, exact time",
 		now:  1984,
 		cfg: &ConfigurationRollout{
-			NextStepTime: 1984,
-			StepSize:     10,
-			StepDuration: 55,
-			Percent:      90,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepSize:     10,
+				StepDuration: 55,
+			},
+			Percent: 90,
 			Revisions: []RevisionRollout{{
 				Percent: 30,
 			}, {
@@ -1211,10 +1269,12 @@ func TestStepRevisions(t *testing.T) {
 			}},
 		},
 		want: &ConfigurationRollout{
-			Percent:      90,
-			NextStepTime: 1984 + 55,
-			StepDuration: 55,
-			StepSize:     10,
+			Percent: 90,
+			StepParams: RolloutParams{
+				NextStepTime: 1984 + 55,
+				StepDuration: 55,
+				StepSize:     10,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 20,
 			}, {
@@ -1225,10 +1285,12 @@ func TestStepRevisions(t *testing.T) {
 		name: "step last, delete some",
 		now:  1988,
 		cfg: &ConfigurationRollout{
-			Percent:      90,
-			NextStepTime: 1984,
-			StepSize:     10,
-			StepDuration: 55,
+			Percent: 90,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepSize:     10,
+				StepDuration: 55,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 22,
 			}, {
@@ -1240,10 +1302,12 @@ func TestStepRevisions(t *testing.T) {
 			}},
 		},
 		want: &ConfigurationRollout{
-			Percent:      90,
-			NextStepTime: 1988 + 55,
-			StepDuration: 55,
-			StepSize:     10,
+			Percent: 90,
+			StepParams: RolloutParams{
+				NextStepTime: 1988 + 55,
+				StepDuration: 55,
+				StepSize:     10,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 20,
 			}, {
@@ -1254,10 +1318,12 @@ func TestStepRevisions(t *testing.T) {
 		name: "over target now",
 		now:  1988,
 		cfg: &ConfigurationRollout{
-			Percent:      15,
-			NextStepTime: 1984,
-			StepSize:     10,
-			StepDuration: 77,
+			Percent: 15,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StepSize:     10,
+				StepDuration: 77,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 5,
 			}, {
@@ -1269,10 +1335,10 @@ func TestStepRevisions(t *testing.T) {
 			}},
 		},
 		want: &ConfigurationRollout{
-			Percent:      15,
-			NextStepTime: 0,
-			StepDuration: 77,
-			StepSize:     10,
+			Percent:    15,
+			StepParams: RolloutParams{
+				// we reset rollout params, since we're done now.
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 15,
 			}},
@@ -1281,11 +1347,13 @@ func TestStepRevisions(t *testing.T) {
 		name: "the very last step",
 		now:  2006,
 		cfg: &ConfigurationRollout{
-			NextStepTime: 1984,
-			Percent:      15,
-			StartTime:    1977,
-			StepDuration: 77,
-			StepSize:     8,
+			Percent: 15,
+			StepParams: RolloutParams{
+				NextStepTime: 1984,
+				StartTime:    1977,
+				StepDuration: 77,
+				StepSize:     8,
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 5,
 			}, {
@@ -1297,11 +1365,10 @@ func TestStepRevisions(t *testing.T) {
 			}},
 		},
 		want: &ConfigurationRollout{
-			NextStepTime: 0,
-			Percent:      15,
-			StartTime:    0,
-			StepDuration: 77,
-			StepSize:     8,
+			Percent:    15,
+			StepParams: RolloutParams{
+				// we reset rollout params, since we're done now.
+			},
 			Revisions: []RevisionRollout{{
 				Percent: 15,
 			}},
diff --git a/third_party/istio-latest/net-istio.yaml b/third_party/istio-latest/net-istio.yaml
index 5412614a3..ace60a975 100644
--- a/third_party/istio-latest/net-istio.yaml
+++ b/third_party/istio-latest/net-istio.yaml
@@ -1,4 +1,4 @@
-# Generated when HEAD was e87b1c73e380aa7798c73cd0a38ec32734af353a
+# Generated when HEAD was b7fe1983bd4d1f1764f7dd83e93157020b0a1dbc
 #
 # Copyright 2019 The Knative Authors
 #
@@ -20,7 +20,7 @@ metadata:
   # These are the permissions needed by the Istio Ingress implementation.
   name: knative-serving-istio
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     serving.knative.dev/controller: "true"
     networking.knative.dev/ingress-provider: istio
 rules:
@@ -50,7 +50,7 @@ metadata:
   name: knative-ingress-gateway
   namespace: knative-serving
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   selector:
@@ -87,7 +87,7 @@ metadata:
   name: knative-local-gateway
   namespace: knative-serving
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   selector:
@@ -106,7 +106,7 @@ metadata:
   name: knative-local-gateway
   namespace: istio-system
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   type: ClusterIP
@@ -126,7 +126,7 @@ metadata:
   name: "webhook"
   namespace: "knative-serving"
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   selector:
@@ -141,7 +141,7 @@ metadata:
   name: "istio-webhook"
   namespace: "knative-serving"
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   selector:
@@ -170,7 +170,7 @@ kind: MutatingWebhookConfiguration
 metadata:
   name: webhook.istio.networking.internal.knative.dev
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 webhooks:
   - admissionReviewVersions:
@@ -206,7 +206,7 @@ kind: ValidatingWebhookConfiguration
 metadata:
   name: config.webhook.istio.networking.internal.knative.dev
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 webhooks:
   - admissionReviewVersions:
@@ -244,7 +244,7 @@ metadata:
   name: istio-webhook-certs
   namespace: knative-serving
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 
 ---
@@ -268,7 +268,7 @@ metadata:
   name: config-istio
   namespace: knative-serving
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 data:
   _example: |
@@ -338,7 +338,7 @@ metadata:
   name: networking-istio
   namespace: knative-serving
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   selector:
@@ -354,14 +354,14 @@ spec:
         sidecar.istio.io/inject: "false"
       labels:
         app: networking-istio
-        serving.knative.dev/release: "v20201208-e87b1c73"
+        serving.knative.dev/release: "v20201209-b7fe1983"
     spec:
       serviceAccountName: controller
       containers:
         - name: networking-istio
           # This is the Go import path for the binary that is containerized
           # and substituted here.
-          image: gcr.io/knative-nightly/knative.dev/net-istio/cmd/controller@sha256:39fde178bac6f7994fe76871fae2c294bce5971627d3874be85176b421749233
+          image: gcr.io/knative-nightly/knative.dev/net-istio/cmd/controller@sha256:d4481e081c6ec6862e0a7cd7cd4c40f5f2021e5488a5e3589063c82be51f8abd
           resources:
             requests:
               cpu: 30m
@@ -418,7 +418,7 @@ metadata:
   name: istio-webhook
   namespace: knative-serving
   labels:
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   selector:
@@ -432,14 +432,14 @@ spec:
       labels:
         app: istio-webhook
         role: istio-webhook
-        serving.knative.dev/release: "v20201208-e87b1c73"
+        serving.knative.dev/release: "v20201209-b7fe1983"
     spec:
       serviceAccountName: controller
       containers:
         - name: webhook
           # This is the Go import path for the binary that is containerized
           # and substituted here.
-          image: gcr.io/knative-nightly/knative.dev/net-istio/cmd/webhook@sha256:38dd61407135120cb3a452914e566e19bdc2fd90499ed1a9f32b824efa605710
+          image: gcr.io/knative-nightly/knative.dev/net-istio/cmd/webhook@sha256:3e873c4895fb1ae766f5d35e9a699fd9612fecca232029f2d453ffc17c8f608f
           resources:
             requests:
               cpu: 20m
@@ -493,7 +493,7 @@ metadata:
   namespace: knative-serving
   labels:
     role: istio-webhook
-    serving.knative.dev/release: "v20201208-e87b1c73"
+    serving.knative.dev/release: "v20201209-b7fe1983"
     networking.knative.dev/ingress-provider: istio
 spec:
   ports: