Repository: kubernetes-sigs/gateway-api

Issue:
Improving distribution tests in conformance
On https://github.com/kubernetes-sigs/gateway-api/pull/3827#discussion_r2123121430

@guicassolato suggested the below: 

> Worth adding some entropy to each iteration to mitigate the risk of coordinated execution perhaps? For example: introducing jitter, randomising the payload, or anything else that varies the sample across iterations (without affecting expected weight distribution).

It would be nice to have something like thig

/good-first-issue

Hints from issue comments:
@k8s-ci-robot: @LiorLieberman: 
	This request has been marked as suitable for new contributors.

### Guidelines
Please ensure that the issue body includes answers to the following questions:
- Why are we solving this issue?
- To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
- Does this issue have zero to low barrier of entry?
- How can the assignee reach out to you for help?


For more details on the requirements of such an issue, please see [here](https://git.k8s.io/community/contributors/guide/help-wanted.md#good-first-issue) and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the `/remove-good-first-issue` command.


<details>

In response to [this](https://github.com/kubernetes-sigs/gateway-api/issues/3832):

>On https://github.com/kubernetes-sigs/gateway-api/pull/3827#discussion_r2123121430
>
>@guicassolato suggested the below: 
>
>> Worth adding some entropy to each iteration to mitigate the risk of coordinated execution perhaps? For example: introducing jitter, randomising the payload, or anything else that varies the sample across iterations (without affecting expected weight distribution).
>
>It would be nice to have something like thig
>
>/good-first-issue


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>@carsontham: /assign@chris-kaiser-7: Can I help with this issue?@LiorLieberman: Sure!
/assign @chris-kaiser-7 @carsontham: hi @LiorLieberman, my apologies for not submitting a PR yet, but I was spending the past few days trying to understand the tests and also running the test locally.

So far, I've ran the tests using the below command, without making any changes ( I wanted to make sure I was running the tests correctly and not break anything ).
```
go test -v ./conformance \
--run TestConformance/MeshHTTPRouteWeight \
--supported-features=Mesh,HTTPRoute
```

And the test is returning me failure even without adding any entropy: 

```
    httproute-weight.go:64: Traffic distribution test failed (10/10): backend "echo-v1" weighted traffic of 0.524 not within tolerance 0.7 (+/-0.050000)
        backend "echo-v2" weighted traffic of 0.476 not within tolerance 0.3 (+/-0.050000)
    httproute-weight.go:69: Weighted distribution tests failed
=== NAME  TestConformance/MeshHTTPRouteWeight
    apply.go:283: 2025-06-10T18:05:28.312926+08:00: Deleting mesh-weighted-backends HTTPRoute
=== NAME  TestConformance
    apply.go:283: 2025-06-10T18:05:28.317196+08:00: Deleting echo-v1 Deployment
    apply.go:283: 2025-06-10T18:05:28.320049+08:00: Deleting gateway-conformance-mesh-consumer Namespace
    apply.go:283: 2025-06-10T18:05:28.322049+08:00: Deleting echo Service
    apply.go:283: 2025-06-10T18:05:28.330848+08:00: Deleting echo-v2 Service
    apply.go:283: 2025-06-10T18:05:28.341497+08:00: Deleting echo-v2 Deployment
    apply.go:283: 2025-06-10T18:05:28.345876+08:00: Deleting echo-v1 Service
    apply.go:283: 2025-06-10T18:05:28.352877+08:00: Deleting echo-v1 Deployment
    apply.go:283: 2025-06-10T18:05:28.357337+08:00: Deleting gateway-conformance-mesh Namespace
--- FAIL: TestConformance (73.35s)
    --- FAIL: TestConformance/MeshHTTPRouteWeight (71.08s)
        --- FAIL: TestConformance/MeshHTTPRouteWeight/Requests_should_have_a_distribution_that_matches_the_weight (71.06s)
```

I'm not sure if I'm running the tests wrongly, but could you point me to the right direction? thanks !@LiorLieberman: @carsontham Sorry! that was last night my time and I did not see that you already assinged. 

Please go ahead, open a PR and I can help you debug it on the PR.

@chris-kaiser-7 - I recommend waiting with this one since @carsontham is already working on it. 

/assign @carsontham @carsontham: hey @LiorLieberman no worries at all! will come up with a PR soon :) Thanks alot!@chris-kaiser-7: /unassign

Solution patch:
diff --git a/conformance/utils/http/http.go b/conformance/utils/http/http.go
index 91ebf225..da7036da 100644
--- a/conformance/utils/http/http.go
+++ b/conformance/utils/http/http.go
@@ -17,7 +17,9 @@ limitations under the License.
 package http
 
 import (
+	"crypto/rand"
 	"fmt"
+	"math/big"
 	"net"
 	"net/url"
 	"strings"
@@ -478,3 +480,54 @@ func setRedirectRequestDefaults(req *roundtripper.Request, cRes *roundtripper.Ca
 		expected.RedirectRequest.Path = req.URL.Path
 	}
 }
+
+// addEntropy adds jitter to the request by adding either a delay up to 1 second, or a random header value, or both.
+func AddEntropy(exp *ExpectedResponse) error {
+	randomNumber := func(limit int64) (*int64, error) {
+		number, err := rand.Int(rand.Reader, big.NewInt(limit))
+		if err != nil {
+			return nil, err
+		}
+		n := number.Int64()
+		return &n, nil
+	}
+
+	// adds a delay
+	delay := func(limit int64) error {
+		randomSleepDuration, err := randomNumber(limit)
+		if err != nil {
+			return err
+		}
+		time.Sleep(time.Duration(*randomSleepDuration) * time.Millisecond)
+		return nil
+	}
+	// adds random header value
+	randomHeader := func(limit int64) error {
+		randomHeaderValue, err := randomNumber(limit)
+		if err != nil {
+			return err
+		}
+		exp.Request.Headers = make(map[string]string)
+		exp.Request.Headers["X-Jitter"] = fmt.Sprintf("%d", *randomHeaderValue)
+		return nil
+	}
+
+	random, err := randomNumber(3)
+	if err != nil {
+		return err
+	}
+
+	switch *random {
+	case 0:
+		return delay(1000)
+	case 1:
+		return randomHeader(10000)
+	case 2:
+		if err := delay(1000); err != nil {
+			return err
+		}
+		return randomHeader(10000)
+	default:
+		return fmt.Errorf("invalid random value: %d", *random)
+	}
+}



Test patch:
diff --git a/conformance/tests/httproute-weight.go b/conformance/tests/httproute-weight.go
index 2c1a520d..fa85deb9 100644
--- a/conformance/tests/httproute-weight.go
+++ b/conformance/tests/httproute-weight.go
@@ -91,7 +91,6 @@ func testDistribution(t *testing.T, suite *suite.ConformanceTestSuite, gwAddr st
 		g               errgroup.Group
 		seenMutex       sync.Mutex
 		seen            = make(map[string]float64, 3 /* number of backends */)
-		req             = http.MakeRequest(t, &expected, gwAddr, "HTTP", "http")
 		expectedWeights = map[string]float64{
 			"infra-backend-v1": 0.7,
 			"infra-backend-v2": 0.3,
@@ -101,6 +100,11 @@ func testDistribution(t *testing.T, suite *suite.ConformanceTestSuite, gwAddr st
 	g.SetLimit(concurrentRequests)
 	for i := 0.0; i < totalRequests; i++ {
 		g.Go(func() error {
+			uniqueExpected := expected
+			if err := http.AddEntropy(&uniqueExpected); err != nil {
+				return fmt.Errorf("error adding entropy: %w", err)
+			}
+			req := http.MakeRequest(t, &uniqueExpected, gwAddr, "HTTP", "http")
 			cReq, cRes, err := roundTripper.CaptureRoundTrip(req)
 			if err != nil {
 				return fmt.Errorf("failed to roundtrip request: %w", err)
diff --git a/conformance/tests/mesh/httproute-weight.go b/conformance/tests/mesh/httproute-weight.go
index cc296d92..844d5471 100644
--- a/conformance/tests/mesh/httproute-weight.go
+++ b/conformance/tests/mesh/httproute-weight.go
@@ -18,16 +18,13 @@ package meshtests
 
 import (
 	"cmp"
-	"crypto/rand"
 	"errors"
 	"fmt"
 	"math"
-	"math/big"
 	"slices"
 	"strings"
 	"sync"
 	"testing"
-	"time"
 
 	"golang.org/x/sync/errgroup"
 
@@ -93,7 +90,7 @@ func testDistribution(t *testing.T, client echo.MeshPod, expected http.ExpectedR
 	for i := 0.0; i < totalRequests; i++ {
 		g.Go(func() error {
 			uniqueExpected := expected
-			if err := addEntropy(&uniqueExpected); err != nil {
+			if err := http.AddEntropy(&uniqueExpected); err != nil {
 				return fmt.Errorf("error adding entropy: %w", err)
 			}
 			_, cRes, err := client.CaptureRequestResponseAndCompare(t, uniqueExpected)
@@ -148,54 +145,3 @@ func testDistribution(t *testing.T, client echo.MeshPod, expected http.ExpectedR
 	})
 	return errors.Join(errs...)
 }
-
-// addEntropy adds jitter to the request by adding either a delay up to 1 second, or a random header value, or both.
-func addEntropy(exp *http.ExpectedResponse) error {
-	randomNumber := func(limit int64) (*int64, error) {
-		number, err := rand.Int(rand.Reader, big.NewInt(limit))
-		if err != nil {
-			return nil, err
-		}
-		n := number.Int64()
-		return &n, nil
-	}
-
-	// adds a delay
-	delay := func(limit int64) error {
-		randomSleepDuration, err := randomNumber(limit)
-		if err != nil {
-			return err
-		}
-		time.Sleep(time.Duration(*randomSleepDuration) * time.Millisecond)
-		return nil
-	}
-	// adds random header value
-	randomHeader := func(limit int64) error {
-		randomHeaderValue, err := randomNumber(limit)
-		if err != nil {
-			return err
-		}
-		exp.Request.Headers = make(map[string]string)
-		exp.Request.Headers["X-Jitter"] = fmt.Sprintf("%d", *randomHeaderValue)
-		return nil
-	}
-
-	random, err := randomNumber(3)
-	if err != nil {
-		return err
-	}
-
-	switch *random {
-	case 0:
-		return delay(1000)
-	case 1:
-		return randomHeader(10000)
-	case 2:
-		if err := delay(1000); err != nil {
-			return err
-		}
-		return randomHeader(10000)
-	default:
-		return fmt.Errorf("invalid random value: %d", *random)
-	}
-}