Repository: knative/serving
Pull request #10378 (2020-12-16 05:01:34)

Update ingress spec to shift traffic based on rollout
- shift traffic when rollout is present
- deep unit tests

/assign @tcnghia 

knative-prow-robot: @vagababov: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
knative-prow-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/knative/serving/pull/10378#" title="Author self-approved">vagababov</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=knative%2Fserving).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[pkg/reconciler/route/OWNERS](https://github.com/knative/serving/blob/master/pkg/reconciler/route/OWNERS)~~ [vagababov]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
knative-prow-robot: @vagababov: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-build-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-build-tests/1339073182555967488) | `/test pull-knative-serving-build-tests`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/10378). [Your PR dashboard](https://gubernator.knative.dev/pr/vagababov).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
knative-prow-robot: @vagababov: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-build-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-build-tests/1339073182555967488) | `/test pull-knative-serving-build-tests`
pull-knative-serving-upgrade-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-upgrade-tests/1339073182614687744) | `/test pull-knative-serving-upgrade-tests`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/10378). [Your PR dashboard](https://gubernator.knative.dev/pr/vagababov).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
knative-prow-robot: @vagababov: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-build-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-build-tests/1339073182555967488) | `/test pull-knative-serving-build-tests`
pull-knative-serving-upgrade-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-upgrade-tests/1339073182614687744) | `/test pull-knative-serving-upgrade-tests`
pull-knative-serving-istio-stable-no-mesh-tls | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-istio-stable-no-mesh-tls/1339073182665019392) | `/test pull-knative-serving-istio-stable-no-mesh-tls`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/10378). [Your PR dashboard](https://gubernator.knative.dev/pr/vagababov).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
knative-prow-robot: @vagababov: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-build-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-build-tests/1339073182555967488) | `/test pull-knative-serving-build-tests`
pull-knative-serving-upgrade-tests | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-upgrade-tests/1339073182614687744) | `/test pull-knative-serving-upgrade-tests`
pull-knative-serving-istio-stable-no-mesh-tls | fb6d2f821f4099b9e8ae8eacb47b041f39a0860f | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-istio-stable-no-mesh-tls/1339073182665019392) | `/test pull-knative-serving-istio-stable-no-mesh-tls`
pull-knative-serving-unit-tests | 82798a3745cecee83193d793cc1ce4debf3fd980 | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-unit-tests/1339073334901477376) | `/test pull-knative-serving-unit-tests`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/10378). [Your PR dashboard](https://gubernator.knative.dev/pr/vagababov).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
knative-prow-robot: @vagababov: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-knative-serving-unit-tests | 82798a3745cecee83193d793cc1ce4debf3fd980 | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-unit-tests/1339073334901477376) | `/test pull-knative-serving-unit-tests`
pull-knative-serving-istio-stable-no-mesh-tls | 82798a3745cecee83193d793cc1ce4debf3fd980 | [link](https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/10378/pull-knative-serving-istio-stable-no-mesh-tls/1339073334968586240) | `/test pull-knative-serving-istio-stable-no-mesh-tls`

[Full PR test history](https://gubernator.knative.dev/pr/knative_serving/10378). [Your PR dashboard](https://gubernator.knative.dev/pr/vagababov).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
vagababov: And fix the table test 😊 
codecov[bot]: # [Codecov](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=h1) Report
> Merging [#10378](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=desc) (253c3d8) into [master](https://codecov.io/gh/knative/serving/commit/69fcc988300a3d64c7f980c0932fcc8fcdd16491?el=desc) (69fcc98) will **increase** coverage by `0.08%`.
> The diff coverage is `97.72%`.

[![Impacted file tree graph](https://codecov.io/gh/knative/serving/pull/10378/graphs/tree.svg?width=650&height=150&src=pr&token=MPpDfAj4Ui)](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #10378      +/-   ##
==========================================
+ Coverage   87.98%   88.06%   +0.08%     
==========================================
  Files         186      186              
  Lines        8738     8764      +26     
==========================================
+ Hits         7688     7718      +30     
+ Misses        810      809       -1     
+ Partials      240      237       -3     
```


| [Impacted Files](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pkg/reconciler/route/resources/ingress.go](https://codecov.io/gh/knative/serving/pull/10378/diff?src=pr&el=tree#diff-cGtnL3JlY29uY2lsZXIvcm91dGUvcmVzb3VyY2VzL2luZ3Jlc3MuZ28=) | `96.68% <97.72%> (-0.12%)` | :arrow_down: |
| [pkg/reconciler/configuration/configuration.go](https://codecov.io/gh/knative/serving/pull/10378/diff?src=pr&el=tree#diff-cGtnL3JlY29uY2lsZXIvY29uZmlndXJhdGlvbi9jb25maWd1cmF0aW9uLmdv) | `88.28% <0.00%> (+3.90%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=footer). Last update [69fcc98...73a22ad](https://codecov.io/gh/knative/serving/pull/10378?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

knative-prow-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/knative/serving/pull/10378#pullrequestreview-553371468" title="LGTM">tcnghia</a>*, *<a href="https://github.com/knative/serving/pull/10378#" title="Author self-approved">vagababov</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=knative%2Fserving).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[pkg/reconciler/route/OWNERS](https://github.com/knative/serving/blob/master/pkg/reconciler/route/OWNERS)~~ [tcnghia,vagababov]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->

Patch:
diff --git a/pkg/reconciler/route/resources/ingress.go b/pkg/reconciler/route/resources/ingress.go
index f7973179f..a2fbfff4c 100644
--- a/pkg/reconciler/route/resources/ingress.go
+++ b/pkg/reconciler/route/resources/ingress.go
@@ -81,10 +81,12 @@ func MakeIngressWithRollout(
 	ingressClass string,
 	acmeChallenges ...netv1alpha1.HTTP01Challenge,
 ) (*netv1alpha1.Ingress, error) {
-	spec, err := makeIngressSpec(ctx, r, tls, tc, acmeChallenges...)
+	spec, err := makeIngressSpec(ctx, r, tls, tc, ro, acmeChallenges...)
 	if err != nil {
 		return nil, err
 	}
+	logger := logging.FromContext(ctx)
+	logger.Infof("@@@ Applying rollout %#v\n", ro)
 	return &netv1alpha1.Ingress{
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      names.Ingress(r),
@@ -122,6 +124,7 @@ func makeIngressSpec(
 	r *servingv1.Route,
 	tls []netv1alpha1.IngressTLS,
 	tc *traffic.Config,
+	ro *traffic.Rollout,
 	acmeChallenges ...netv1alpha1.HTTP01Challenge,
 ) (netv1alpha1.IngressSpec, error) {
 	// Domain should have been specified in route status
@@ -149,7 +152,8 @@ func makeIngressSpec(
 			if err != nil {
 				return netv1alpha1.IngressSpec{}, err
 			}
-			rule := makeIngressRule(domains, r.Namespace, visibility, tc.Targets[name])
+			rule := makeIngressRule(domains, r.Namespace, name,
+				visibility, tc.Targets[name], ro)
 			if featuresConfig.TagHeaderBasedRouting == apicfg.Enabled {
 				if rule.HTTP.Paths[0].AppendHeaders == nil {
 					rule.HTTP.Paths[0].AppendHeaders = make(map[string]string, 1)
@@ -171,7 +175,7 @@ func makeIngressSpec(
 					// Since names are sorted `DefaultTarget == ""` is the first one,
 					// so just pass the subslice.
 					rule.HTTP.Paths = append(
-						makeTagBasedRoutingIngressPaths(r.Namespace, tc, names[1:]), rule.HTTP.Paths...)
+						makeTagBasedRoutingIngressPaths(r.Namespace, tc, ro, names[1:]), rule.HTTP.Paths...)
 				} else {
 					// If a request is routed by a tag-attached hostname instead of the tag header,
 					// the request may not have the tag header "Knative-Serving-Tag",
@@ -252,25 +256,27 @@ func makeACMEIngressPaths(challenges map[string]netv1alpha1.HTTP01Challenge, dom
 	return paths
 }
 
-func makeIngressRule(domains []string, ns string,
-	visibility netv1alpha1.IngressVisibility, targets traffic.RevisionTargets) netv1alpha1.IngressRule {
+func makeIngressRule(domains []string, ns, tag string,
+	visibility netv1alpha1.IngressVisibility,
+	targets traffic.RevisionTargets,
+	ro *traffic.Rollout) netv1alpha1.IngressRule {
 	return netv1alpha1.IngressRule{
 		Hosts:      domains,
 		Visibility: visibility,
 		HTTP: &netv1alpha1.HTTPIngressRuleValue{
 			Paths: []netv1alpha1.HTTPIngressPath{
-				*makeBaseIngressPath(ns, targets),
+				*makeBaseIngressPath(ns, tag, targets, ro),
 			},
 		},
 	}
 }
 
 // `names` should not include `""` — the DefaultTarget.
-func makeTagBasedRoutingIngressPaths(ns string, tc *traffic.Config, names []string) []netv1alpha1.HTTPIngressPath {
+func makeTagBasedRoutingIngressPaths(ns string, tc *traffic.Config, ro *traffic.Rollout, names []string) []netv1alpha1.HTTPIngressPath {
 	paths := make([]netv1alpha1.HTTPIngressPath, 0, len(names))
 
 	for _, name := range names {
-		path := makeBaseIngressPath(ns, tc.Targets[name])
+		path := makeBaseIngressPath(ns, name, tc.Targets[name], ro)
 		path.Headers = map[string]netv1alpha1.HeaderMatch{network.TagHeaderName: {Exact: name}}
 		paths = append(paths, *path)
 	}
@@ -278,28 +284,66 @@ func makeTagBasedRoutingIngressPaths(ns string, tc *traffic.Config, names []stri
 	return paths
 }
 
-func makeBaseIngressPath(ns string, targets traffic.RevisionTargets) *netv1alpha1.HTTPIngressPath {
+func rolloutConfig(cfgName string, ros []*traffic.ConfigurationRollout) *traffic.ConfigurationRollout {
+	for _, ro := range ros {
+		if ro.ConfigurationName == cfgName {
+			return ro
+		}
+	}
+	// Technically impossible with valid inputs.
+	return nil
+}
+
+func makeBaseIngressPath(ns, tag string, targets traffic.RevisionTargets,
+	ro *traffic.Rollout) *netv1alpha1.HTTPIngressPath {
+	roCfgs := ro.RolloutsByTag(tag)
+
 	// Optimistically allocate |targets| elements.
 	splits := make([]netv1alpha1.IngressBackendSplit, 0, len(targets))
 	for _, t := range targets {
+		var cfg *traffic.ConfigurationRollout
 		if *t.Percent == 0 {
 			continue
 		}
 
-		splits = append(splits, netv1alpha1.IngressBackendSplit{
-			IngressBackend: netv1alpha1.IngressBackend{
-				ServiceNamespace: ns,
-				ServiceName:      t.ServiceName,
-				// Port on the public service must match port on the activator.
-				// Otherwise, the serverless services can't guarantee seamless positive handoff.
-				ServicePort: intstr.FromInt(networking.ServicePort(t.Protocol)),
-			},
-			Percent: int(*t.Percent),
-			AppendHeaders: map[string]string{
-				activator.RevisionHeaderName:      t.TrafficTarget.RevisionName,
-				activator.RevisionHeaderNamespace: ns,
-			},
-		})
+		if t.LatestRevision != nil && *t.LatestRevision {
+			cfg = rolloutConfig(t.ConfigurationName, roCfgs)
+		}
+		if cfg == nil || len(cfg.Revisions) < 2 {
+			// No rollout in progress.
+			splits = append(splits, netv1alpha1.IngressBackendSplit{
+				IngressBackend: netv1alpha1.IngressBackend{
+					ServiceNamespace: ns,
+					ServiceName:      t.ServiceName,
+					// Port on the public service must match port on the activator.
+					// Otherwise, the serverless services can't guarantee seamless positive handoff.
+					ServicePort: intstr.FromInt(networking.ServicePort(t.Protocol)),
+				},
+				Percent: int(*t.Percent),
+				AppendHeaders: map[string]string{
+					activator.RevisionHeaderName:      t.TrafficTarget.RevisionName,
+					activator.RevisionHeaderNamespace: ns,
+				},
+			})
+		} else {
+			for i := range cfg.Revisions {
+				rev := &cfg.Revisions[i]
+				splits = append(splits, netv1alpha1.IngressBackendSplit{
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      rev.RevisionName,
+						// Port on the public service must match port on the activator.
+						// Otherwise, the serverless services can't guarantee seamless positive handoff.
+						ServicePort: intstr.FromInt(networking.ServicePort(t.Protocol)),
+					},
+					Percent: rev.Percent,
+					AppendHeaders: map[string]string{
+						activator.RevisionHeaderName:      rev.RevisionName,
+						activator.RevisionHeaderNamespace: ns,
+					},
+				})
+			}
+		}
 	}
 
 	return &netv1alpha1.HTTPIngressPath{


Test patch:
diff --git a/pkg/reconciler/route/resources/ingress_test.go b/pkg/reconciler/route/resources/ingress_test.go
index 3e773ee17..6417b394d 100644
--- a/pkg/reconciler/route/resources/ingress_test.go
+++ b/pkg/reconciler/route/resources/ingress_test.go
@@ -95,11 +95,9 @@ func TestMakeIngressCorrectMetadata(t *testing.T) {
 	}
 }
 
-func TestMakeIngressWithRollout(t *testing.T) {
-	const (
-		ingressClass         = "ng-ingress"
-		passdownIngressClass = "ok-ingress"
-	)
+func TestMakeIngressWithTaggedRollout(t *testing.T) {
+	const ingressClass = "ng-ingress"
+
 	cfg := &traffic.Config{
 		Targets: map[string]traffic.RevisionTargets{
 			"tagged": {{
@@ -127,10 +125,11 @@ func TestMakeIngressWithRollout(t *testing.T) {
 		serving.RouteNamespaceLabelKey: "try-to-override",
 		"test-label":                   "foo",
 	}), WithRouteAnnotation(map[string]string{
-		networking.IngressClassAnnotationKey: passdownIngressClass,
+		networking.IngressClassAnnotationKey: ingressClass,
 		"test-annotation":                    "bar",
 	}), WithRouteUID("1234-5678"), WithURL)
-	expected := metav1.ObjectMeta{
+
+	wantMeta := metav1.ObjectMeta{
 		Name:      "test-route",
 		Namespace: ns,
 		Labels: map[string]string{
@@ -139,8 +138,7 @@ func TestMakeIngressWithRollout(t *testing.T) {
 			"test-label":                   "foo",
 		},
 		Annotations: map[string]string{
-			// Make sure to get passdownIngressClass instead of ingressClass
-			networking.IngressClassAnnotationKey: passdownIngressClass,
+			networking.IngressClassAnnotationKey: ingressClass,
 			networking.RolloutAnnotationKey:      `{"configurations":[{"configurationName":"valhalla","percent":100,"revisions":[{"revisionName":"valhalla-01982","percent":100}],"stepParams":{}},{"configurationName":"thor","tag":"tagged","percent":100,"revisions":[{"revisionName":"thor-02020","percent":100}],"stepParams":{}}]}`,
 			"test-annotation":                    "bar",
 		},
@@ -151,8 +149,318 @@ func TestMakeIngressWithRollout(t *testing.T) {
 		t.Error("Unexpected error", err)
 	}
 
-	if !cmp.Equal(expected, ing.ObjectMeta) {
-		t.Error("Unexpected metadata (-want, +got):", cmp.Diff(expected, ing.ObjectMeta))
+	if !cmp.Equal(wantMeta, ing.ObjectMeta) {
+		t.Error("Unexpected metadata (-want, +got):", cmp.Diff(wantMeta, ing.ObjectMeta))
+	}
+}
+
+func TestMakeIngressWithActualRollout(t *testing.T) {
+	const ingressClass = "ng-ingress"
+	ro := &traffic.Rollout{
+		Configurations: []traffic.ConfigurationRollout{{
+			ConfigurationName: "rune",
+			Percent:           1,
+			Revisions: []traffic.RevisionRollout{{
+				RevisionName: "rune-01911",
+				Percent:      1,
+			}},
+		}, {
+			ConfigurationName: "valhalla",
+			Percent:           99,
+			Revisions: []traffic.RevisionRollout{{
+				RevisionName: "valhalla-01981",
+				Percent:      41,
+			}, {
+				RevisionName: "valhalla-01982",
+				Percent:      68,
+			}},
+		}, {
+			ConfigurationName: "rune",
+			Percent:           1,
+			Revisions: []traffic.RevisionRollout{{
+				RevisionName: "rune-01911",
+				Percent:      1,
+			}},
+		}, {
+			ConfigurationName: "thor",
+			Tag:               "hammer",
+			Percent:           80,
+			Revisions: []traffic.RevisionRollout{{
+				RevisionName: "thor-02018",
+				Percent:      60,
+			}, {
+				RevisionName: "thor-02019",
+				Percent:      15,
+			}, {
+				RevisionName: "thor-02020",
+				Percent:      5,
+			}},
+		}},
+	}
+	cfg := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			"hammer": {{
+				TrafficTarget: v1.TrafficTarget{
+					ConfigurationName: "thor",
+					LatestRevision:    ptr.Bool(true),
+					Percent:           ptr.Int64(80),
+					RevisionName:      "thor-02020",
+				},
+				ServiceName: "thor-02020",
+			}, {
+				TrafficTarget: v1.TrafficTarget{
+					ConfigurationName: "thor",
+					LatestRevision:    ptr.Bool(false),
+					Percent:           ptr.Int64(20),
+					RevisionName:      "thor-beta",
+				},
+				ServiceName: "thor-beta",
+			}},
+			traffic.DefaultTarget: {{
+				TrafficTarget: v1.TrafficTarget{
+					ConfigurationName: "rune",
+					LatestRevision:    ptr.Bool(true),
+					Percent:           ptr.Int64(1),
+					RevisionName:      "rune-01911",
+				},
+				ServiceName: "rune-01911",
+			}, {
+				TrafficTarget: v1.TrafficTarget{
+					ConfigurationName: "valhalla",
+					LatestRevision:    ptr.Bool(true),
+					Percent:           ptr.Int64(99),
+					RevisionName:      "valhalla-01982",
+				},
+				ServiceName: "valhalla-01982",
+			}},
+		},
+	}
+	r := Route(ns, "test-route", WithRouteAnnotation(map[string]string{
+		networking.IngressClassAnnotationKey: ingressClass,
+	}), WithRouteUID("1234-5678"), WithURL)
+
+	wantMeta := metav1.ObjectMeta{
+		Name:      "test-route",
+		Namespace: ns,
+		Labels: map[string]string{
+			serving.RouteLabelKey:          "test-route",
+			serving.RouteNamespaceLabelKey: ns,
+		},
+		Annotations: map[string]string{
+			networking.IngressClassAnnotationKey: ingressClass,
+			networking.RolloutAnnotationKey:      serializeRollout(context.Background(), ro),
+		},
+		OwnerReferences: []metav1.OwnerReference{*kmeta.NewControllerRef(r)},
+	}
+	ing, err := MakeIngressWithRollout(testContext(), r, cfg, ro, nil /*tls*/, ingressClass)
+	if err != nil {
+		t.Error("Unexpected error", err)
+	}
+
+	if !cmp.Equal(wantMeta, ing.ObjectMeta) {
+		t.Error("Unexpected metadata (-want, +got):", cmp.Diff(wantMeta, ing.ObjectMeta))
+	}
+	wantRules := []netv1alpha1.IngressRule{{
+		Hosts: []string{
+			"test-route." + ns,
+			"test-route." + ns + ".svc",
+			pkgnet.GetServiceHostname("test-route", ns),
+		},
+		HTTP: &netv1alpha1.HTTPIngressRuleValue{
+			Paths: []netv1alpha1.HTTPIngressPath{{
+				Splits: []netv1alpha1.IngressBackendSplit{{
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "rune-01911",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 1,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "rune-01911",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "valhalla-01981",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 41,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "valhalla-01981",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "valhalla-01982",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 68,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "valhalla-01982",
+						"Knative-Serving-Namespace": ns,
+					},
+				}},
+			}},
+		},
+		Visibility: netv1alpha1.IngressVisibilityClusterLocal,
+	}, {
+		Hosts: []string{
+			"test-route." + ns + ".example.com",
+		},
+		HTTP: &netv1alpha1.HTTPIngressRuleValue{
+			Paths: []netv1alpha1.HTTPIngressPath{{
+				Splits: []netv1alpha1.IngressBackendSplit{{
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "rune-01911",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 1,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "rune-01911",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "valhalla-01981",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 41,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "valhalla-01981",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "valhalla-01982",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 68,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "valhalla-01982",
+						"Knative-Serving-Namespace": ns,
+					},
+				}},
+			}},
+		},
+		Visibility: netv1alpha1.IngressVisibilityExternalIP,
+	}, {
+		Hosts: []string{
+			"hammer-test-route." + ns,
+			"hammer-test-route." + ns + ".svc",
+			pkgnet.GetServiceHostname("hammer-test-route", ns),
+		},
+		HTTP: &netv1alpha1.HTTPIngressRuleValue{
+			Paths: []netv1alpha1.HTTPIngressPath{{
+				Splits: []netv1alpha1.IngressBackendSplit{{
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-02018",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 60,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-02018",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-02019",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 15,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-02019",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-02020",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 5,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-02020",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-beta",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 20,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-beta",
+						"Knative-Serving-Namespace": ns,
+					},
+				}},
+			}},
+		},
+		Visibility: netv1alpha1.IngressVisibilityClusterLocal,
+	}, {
+		Hosts: []string{
+			"hammer-test-route." + ns + ".example.com",
+		},
+		HTTP: &netv1alpha1.HTTPIngressRuleValue{
+			Paths: []netv1alpha1.HTTPIngressPath{{
+				Splits: []netv1alpha1.IngressBackendSplit{{
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-02018",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 60,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-02018",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-02019",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 15,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-02019",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-02020",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 5,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-02020",
+						"Knative-Serving-Namespace": ns,
+					},
+				}, {
+					IngressBackend: netv1alpha1.IngressBackend{
+						ServiceNamespace: ns,
+						ServiceName:      "thor-beta",
+						ServicePort:      intstr.FromInt(80),
+					},
+					Percent: 20,
+					AppendHeaders: map[string]string{
+						"Knative-Serving-Revision":  "thor-beta",
+						"Knative-Serving-Namespace": ns,
+					},
+				}},
+			}},
+		},
+		Visibility: netv1alpha1.IngressVisibilityExternalIP,
+	}}
+	if got, want := ing.Spec.Rules, wantRules; !cmp.Equal(got, want) {
+		t.Errorf("Rules mismatch: diff(-want,+got)\n%s", cmp.Diff(want, got))
 	}
 }
 
@@ -283,7 +591,9 @@ func TestMakeIngressSpecCorrectRules(t *testing.T) {
 		Visibility: netv1alpha1.IngressVisibilityExternalIP,
 	}}
 
-	ci, err := makeIngressSpec(testContext(), r, nil, &traffic.Config{Targets: targets})
+	tc := &traffic.Config{Targets: targets}
+	ro := tc.BuildRollout()
+	ci, err := makeIngressSpec(testContext(), r, nil /*tls*/, tc, ro)
 	if err != nil {
 		t.Error("Unexpected error", err)
 	}
@@ -356,20 +666,22 @@ func TestMakeIngressSpecCorrectRuleVisibility(t *testing.T) {
 	}}
 	for _, c := range cases {
 		t.Run(c.name, func(t *testing.T) {
-			ci, err := makeIngressSpec(testContext(), c.route, nil, &traffic.Config{
+			tc := &traffic.Config{
 				Targets:    c.targets,
 				Visibility: c.serviceVisibility,
-			})
+			}
+			ro := tc.BuildRollout()
+			ci, err := makeIngressSpec(testContext(), c.route, nil /*tls*/, tc, ro)
 			if err != nil {
 				t.Error("Unexpected error", err)
 			}
 			if len(c.expectedVisibility) != len(ci.Rules) {
-				t.Errorf("Unexpected %d rules, saw %d", len(c.expectedVisibility), len(ci.Rules))
+				t.Errorf("|rules| = %d, want: %d", len(ci.Rules), len(c.expectedVisibility))
 			}
 			for _, rule := range ci.Rules {
 				visibility := rule.Visibility
 				if !cmp.Equal(c.expectedVisibility[visibility], rule.Hosts) {
-					t.Errorf("Expected hosts %s for visibility %s, saw %s", c.expectedVisibility, visibility, rule.Hosts)
+					t.Errorf("Hosts for visibility[%s] = %v, want: %v", visibility, rule.Hosts, c.expectedVisibility)
 				}
 			}
 		})
@@ -539,7 +851,9 @@ func TestMakeIngressSpecCorrectRulesWithTagBasedRouting(t *testing.T) {
 	ctx := testContext()
 	config.FromContext(ctx).Features.TagHeaderBasedRouting = apicfg.Enabled
 
-	ci, err := makeIngressSpec(ctx, r, nil /*tls*/, &traffic.Config{Targets: targets})
+	tc := &traffic.Config{Targets: targets}
+	ro := tc.BuildRollout()
+	ci, err := makeIngressSpec(ctx, r, nil /*tls*/, tc, ro)
 	if err != nil {
 		t.Error("Unexpected error", err)
 	}
@@ -551,7 +865,8 @@ func TestMakeIngressSpecCorrectRulesWithTagBasedRouting(t *testing.T) {
 
 // One active target.
 func TestMakeIngressRuleVanilla(t *testing.T) {
-	targets := []traffic.RevisionTarget{{
+	domains := []string{"a.com", "b.org"}
+	targets := traffic.RevisionTargets{{
 		TrafficTarget: v1.TrafficTarget{
 			ConfigurationName: "config",
 			RevisionName:      "revision",
@@ -559,8 +874,14 @@ func TestMakeIngressRuleVanilla(t *testing.T) {
 		},
 		ServiceName: "chocolate",
 	}}
-	domains := []string{"a.com", "b.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			traffic.DefaultTarget: targets,
+		},
+	}
+	ro := tc.BuildRollout()
+	rule := makeIngressRule(domains, ns, traffic.DefaultTarget,
+		netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{
 			"a.com",
@@ -607,7 +928,14 @@ func TestMakeIngressRuleZeroPercentTarget(t *testing.T) {
 		},
 	}}
 	domains := []string{"test.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			traffic.DefaultTarget: targets,
+		},
+	}
+	ro := tc.BuildRollout()
+	rule := makeIngressRule(domains, ns, traffic.DefaultTarget,
+		netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{"test.org"},
 		HTTP: &netv1alpha1.HTTPIngressRuleValue{
@@ -651,8 +979,14 @@ func TestMakeIngressRuleTwoTargets(t *testing.T) {
 		},
 		ServiceName: "death",
 	}}
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			"a-tag": targets,
+		},
+	}
+	ro := tc.BuildRollout()
 	domains := []string{"test.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	rule := makeIngressRule(domains, ns, "a-tag", netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{"test.org"},
 		HTTP: &netv1alpha1.HTTPIngressRuleValue{
@@ -686,7 +1020,7 @@ func TestMakeIngressRuleTwoTargets(t *testing.T) {
 	}
 
 	if !cmp.Equal(expected, rule) {
-		t.Error("Unexpected rule (-want, +got):", cmp.Diff(expected, rule))
+		t.Errorf("Unexpected rule (-want, +got):\n%s", cmp.Diff(expected, rule))
 	}
 }
 
@@ -700,8 +1034,15 @@ func TestMakeIngressRuleInactiveTarget(t *testing.T) {
 		},
 		ServiceName: "strange-quark",
 	}}
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			traffic.DefaultTarget: targets,
+		},
+	}
+	ro := tc.BuildRollout()
 	domains := []string{"a.com", "b.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	rule := makeIngressRule(domains, ns, traffic.DefaultTarget,
+		netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{
 			"a.com",
@@ -747,8 +1088,15 @@ func TestMakeIngressRuleTwoInactiveTargets(t *testing.T) {
 		},
 		ServiceName: "down-quark",
 	}}
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			traffic.DefaultTarget: targets,
+		},
+	}
+	ro := tc.BuildRollout()
 	domains := []string{"a.com", "b.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	rule := makeIngressRule(domains, ns, traffic.DefaultTarget,
+		netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{
 			"a.com",
@@ -803,8 +1151,15 @@ func TestMakeIngressRuleZeroPercentTargetInactive(t *testing.T) {
 			Percent:           ptr.Int64(0),
 		},
 	}}
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			traffic.DefaultTarget: targets,
+		},
+	}
+	ro := tc.BuildRollout()
 	domains := []string{"test.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	rule := makeIngressRule(domains, ns, traffic.DefaultTarget,
+		netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{"test.org"},
 		HTTP: &netv1alpha1.HTTPIngressRuleValue{
@@ -846,8 +1201,15 @@ func TestMakeIngressRuleImplicit0PercentTargetInactive(t *testing.T) {
 			Percent:           ptr.Int64(0),
 		},
 	}}
+	tc := &traffic.Config{
+		Targets: map[string]traffic.RevisionTargets{
+			traffic.DefaultTarget: targets,
+		},
+	}
+	ro := tc.BuildRollout()
 	domains := []string{"test.org"}
-	rule := makeIngressRule(domains, ns, netv1alpha1.IngressVisibilityExternalIP, targets)
+	rule := makeIngressRule(domains, ns, traffic.DefaultTarget,
+		netv1alpha1.IngressVisibilityExternalIP, targets, ro)
 	expected := netv1alpha1.IngressRule{
 		Hosts: []string{"test.org"},
 		HTTP: &netv1alpha1.HTTPIngressRuleValue{
@@ -1026,7 +1388,12 @@ func TestMakeIngressACMEChallenges(t *testing.T) {
 			}}},
 	}}
 
-	ci, err := makeIngressSpec(testContext(), r, nil, &traffic.Config{Targets: targets}, acmeChallenge)
+	tc := &traffic.Config{
+		Targets: targets,
+	}
+	ro := tc.BuildRollout()
+
+	ci, err := makeIngressSpec(testContext(), r, nil /*tls*/, tc, ro, acmeChallenge)
 	if err != nil {
 		t.Error("Unexpected error", err)
 	}
diff --git a/pkg/reconciler/route/table_test.go b/pkg/reconciler/route/table_test.go
index 62dd72edd..2e28f905d 100644
--- a/pkg/reconciler/route/table_test.go
+++ b/pkg/reconciler/route/table_test.go
@@ -418,7 +418,7 @@ func TestReconcile(t *testing.T) {
 		},
 		WantUpdates: []clientgotesting.UpdateActionImpl{{
 			// ingress should be updated with the new rollout data.
-			Object: simpleReadyIngress(
+			Object: ingressWithRollout(
 				Route("default", "becomes-ready", WithConfigTarget("config"), WithURL),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
@@ -432,6 +432,18 @@ func TestReconcile(t *testing.T) {
 						}},
 					},
 				},
+				&traffic.Rollout{
+					Configurations: []traffic.ConfigurationRollout{{
+						ConfigurationName: "config",
+						Revisions: []traffic.RevisionRollout{{
+							RevisionName: "config-00000",
+							Percent:      99,
+						}, {
+							RevisionName: "config-00001",
+							Percent:      1,
+						}},
+					}},
+				},
 				simpleRollout("config", []traffic.RevisionRollout{{
 					RevisionName: "config-00000", Percent: 99,
 				}, {
@@ -784,7 +796,7 @@ func TestReconcile(t *testing.T) {
 		},
 		// A new LatestReadyRevisionName on the Configuration should result in the new Revision being rolled out.
 		WantUpdates: []clientgotesting.UpdateActionImpl{{
-			Object: simpleReadyIngress(
+			Object: ingressWithRollout(
 				Route("default", "new-latest-ready", WithConfigTarget("config"), WithURL, WithRouteGeneration(1)),
 				&traffic.Config{
 					Targets: map[string]traffic.RevisionTargets{
@@ -800,6 +812,18 @@ func TestReconcile(t *testing.T) {
 						}},
 					},
 				},
+				&traffic.Rollout{
+					Configurations: []traffic.ConfigurationRollout{{
+						ConfigurationName: "config",
+						Revisions: []traffic.RevisionRollout{{
+							RevisionName: "config-00001",
+							Percent:      99,
+						}, {
+							RevisionName: "config-00002",
+							Percent:      1,
+						}},
+					}},
+				},
 				simpleRollout("config", []traffic.RevisionRollout{{
 					RevisionName: "config-00001", Percent: 99,
 				}, {
@@ -2644,6 +2668,15 @@ func simpleReadyIngress(r *v1.Route, tc *traffic.Config, io ...IngressOption) *n
 	return ingress
 }
 
+func ingressWithRollout(r *v1.Route, tc *traffic.Config, ro *traffic.Rollout, io ...IngressOption) *netv1alpha1.Ingress {
+	ingress, _ := resources.MakeIngressWithRollout(getContext(), r, tc, ro, nil /*tls*/, TestIngressClass)
+	ingress.Status = readyIngressStatus()
+	for _, o := range io {
+		o(ingress)
+	}
+	return ingress
+}
+
 func readyIngressStatus() netv1alpha1.IngressStatus {
 	status := netv1alpha1.IngressStatus{}
 	status.InitializeConditions()