Repository: kubernetes-sigs/kueue

Issue:
Flaky E2E Test: ManageJobsWithoutQueueName when manageJobsWithoutQueueName=true should suspend the pods created by a StatefulSet in the test namespace without queue-name label
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!

If the matter is security related, please disclose it privately via https://kubernetes.io/security/
-->


**What happened**:

`End To End Custom Configs handling Suite: kindest/node:v1.34.0: [It] ManageJobsWithoutQueueName when manageJobsWithoutQueueName=true should suspend the pods created by a StatefulSet in the test namespace without queue-name label` failed in https://github.com/kubernetes-sigs/kueue/pull/6686

```
{Timed out after 45.000s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:648 with:
Expected success, but got an error:
    <*errors.StatusError | 0xc0008886e0>: 
    workloads.kueue.x-k8s.io "statefulset-sts-18b6e" not found
    {
        ErrStatus: {
            TypeMeta: {Kind: "", APIVersion: ""},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            },
            Status: "Failure",
            Message: "workloads.kueue.x-k8s.io \"statefulset-sts-18b6e\" not found",
            Reason: "NotFound",
            Details: {
                Name: "statefulset-sts-18b6e",
                Group: "kueue.x-k8s.io",
                Kind: "workloads",
                UID: "",
                Causes: nil,
                RetryAfterSeconds: 0,
            },
            Code: 404,
        },
    } failed [FAILED] Timed out after 45.000s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:648 with:
Expected success, but got an error:
    <*errors.StatusError | 0xc0008886e0>: 
    workloads.kueue.x-k8s.io "statefulset-sts-18b6e" not found
    {
        ErrStatus: {
            TypeMeta: {Kind: "", APIVersion: ""},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            },
            Status: "Failure",
            Message: "workloads.kueue.x-k8s.io \"statefulset-sts-18b6e\" not found",
            Reason: "NotFound",
            Details: {
                Name: "statefulset-sts-18b6e",
                Group: "kueue.x-k8s.io",
                Kind: "workloads",
                UID: "",
                Causes: nil,
                RetryAfterSeconds: 0,
            },
            Code: 404,
        },
    }
In [It] at: /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:650 @ 08/29/25 19:02:47.399
}
```

**What you expected to happen**:

No errors.

**How to reproduce it (as minimally and precisely as possible)**:

https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/6686/pull-kueue-test-e2e-customconfigs-main/1961500167513313280

**Anything else we need to know?**:

**Environment**:
- Kubernetes version (use `kubectl version`):
- Kueue version (use `git describe --tags --dirty --always`):
- Cloud provider or hardware configuration:
- OS (e.g: `cat /etc/os-release`):
- Kernel (e.g. `uname -a`):
- Install tools:
- Others:

Discussion:
@tenzen-y: /kind flake
@IrvingMg: /assign@mimowo: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/6765/pull-kueue-test-e2e-customconfigs-main/1971185466099109888@tenzen-y: This happened again.

https://github.com/kubernetes-sigs/kueue/pull/7356
https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/7356/pull-kueue-test-e2e-customconfigs-main/1981316427394584576

/reopen@k8s-ci-robot: @tenzen-y: Reopened this issue.

<details>

In response to [this](https://github.com/kubernetes-sigs/kueue/issues/6699#issuecomment-3436447942):

>This happened again.
>
>https://github.com/kubernetes-sigs/kueue/pull/7356
>https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/7356/pull-kueue-test-e2e-customconfigs-main/1981316427394584576
>
>/reopen


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>@mimowo: It is the same test, but different assert, and I think a different root cause
```
{Timed out after 45.000s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:657 with:
Expected
    <int32>: 0
to equal
    <int32>: 3 failed [FAILED] Timed out after 45.000s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:657 with:
Expected
    <int32>: 0
to equal
    <int32>: 3
In [It] at: /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:658 @ 10/23/25 11:18:28.239
}
```@tenzen-y: > It is the same test, but different assert, and I think a different root cause
> 
> ```
> {Timed out after 45.000s.
> The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:657 with:
> Expected
>     <int32>: 0
> to equal
>     <int32>: 3 failed [FAILED] Timed out after 45.000s.
> The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:657 with:
> Expected
>     <int32>: 0
> to equal
>     <int32>: 3
> In [It] at: /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/customconfigs/managejobswithoutqueuename_test.go:658 @ 10/23/25 11:18:28.239
> }
> ```

Good point, let me open another issue, thanks.@tenzen-y: /close
@k8s-ci-robot: @tenzen-y: Closing this issue.

<details>

In response to [this](https://github.com/kubernetes-sigs/kueue/issues/6699#issuecomment-3436632531):

>/close
>


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>@tenzen-y: Opened https://github.com/kubernetes-sigs/kueue/issues/7366

Solution patch:
diff --git a/pkg/controller/jobs/pod/event_handlers.go b/pkg/controller/jobs/pod/event_handlers.go
index 629375a01..2af453dde 100644
--- a/pkg/controller/jobs/pod/event_handlers.go
+++ b/pkg/controller/jobs/pod/event_handlers.go
@@ -126,7 +126,8 @@ func (h *workloadHandler) Update(ctx context.Context, e event.UpdateEvent, q wor
 	h.queueReconcileForChildPod(ctx, e.ObjectNew, q)
 }
 
-func (h *workloadHandler) Delete(context.Context, event.DeleteEvent, workqueue.TypedRateLimitingInterface[reconcile.Request]) {
+func (h *workloadHandler) Delete(ctx context.Context, e event.DeleteEvent, q workqueue.TypedRateLimitingInterface[reconcile.Request]) {
+	h.queueReconcileForChildPod(ctx, e.Object, q)
 }
 
 func (h *workloadHandler) Generic(_ context.Context, _ event.GenericEvent, _ workqueue.TypedRateLimitingInterface[reconcile.Request]) {
diff --git a/pkg/scheduler/flavorassigner/flavorassigner.go b/pkg/scheduler/flavorassigner/flavorassigner.go
index 9af7a5fb6..eb0928fe6 100644
--- a/pkg/scheduler/flavorassigner/flavorassigner.go
+++ b/pkg/scheduler/flavorassigner/flavorassigner.go
@@ -19,6 +19,7 @@ package flavorassigner
 import (
 	"fmt"
 	"maps"
+	"math"
 	"slices"
 	"sort"
 	"strconv"
@@ -321,12 +322,39 @@ func (m FlavorAssignmentMode) String() string {
 	return "Unknown"
 }
 
+// borrowingLevel represents how locally the quota can be sourced. 0
+// indicates that quota is available within the ClusterQueue, while
+// progressively higher numbers indicate capacity comes from a more
+// distant cohort.  Please note that while this number is
+// monotonically increasing, it is not necessarily sequential.
+type borrowingLevel int
+
+// betterThan indicates that Flavor represented by b has NominalQuota
+// available more locally than the flavor represented by other.
+func (b borrowingLevel) betterThan(other borrowingLevel) bool {
+	return b < other
+}
+
+// optimal indicates that capacity is available at the ClusterQueue
+// level, i.e. no borrowing.
+func (b borrowingLevel) optimal() bool {
+	return b == 0
+}
+
 // granularMode is the FlavorAssignmentMode internal to
-// FlavorAssigner, which lets us distinguish priority based preemption,
-// reclamation within Cohort and borrowing.
+// FlavorAssigner, which lets us distinguish priority based
+// preemption, reclamation within Cohort and borrowing.
 type granularMode struct {
 	preemptionMode preemptionMode
-	needsBorrowing bool
+	borrowingLevel borrowingLevel
+}
+
+func worstGranularMode() granularMode {
+	return granularMode{preemptionMode: noFit, borrowingLevel: math.MaxInt}
+}
+
+func bestGranularMode() granularMode {
+	return granularMode{preemptionMode: fit, borrowingLevel: 0}
 }
 
 type preemptionMode int
@@ -354,20 +382,20 @@ func isPreferred(a, b granularMode, fungibilityConfig kueue.FlavorFungibility) b
 		if a.preemptionMode != b.preemptionMode {
 			return a.preemptionMode > b.preemptionMode
 		} else {
-			return !a.needsBorrowing && b.needsBorrowing
+			return a.borrowingLevel.betterThan(b.borrowingLevel)
 		}
 	}
 
 	if fungibilityConfig.WhenCanBorrow == kueue.TryNextFlavor {
-		if a.needsBorrowing != b.needsBorrowing {
-			return !a.needsBorrowing
+		if a.borrowingLevel != b.borrowingLevel {
+			return a.borrowingLevel.betterThan(b.borrowingLevel)
 		}
 		return a.preemptionMode > b.preemptionMode
 	} else {
 		if a.preemptionMode != b.preemptionMode {
 			return a.preemptionMode > b.preemptionMode
 		}
-		return !a.needsBorrowing && b.needsBorrowing
+		return a.borrowingLevel.betterThan(b.borrowingLevel)
 	}
 }
 
@@ -664,7 +692,7 @@ func (a *FlavorAssigner) findFlavorForPodSets(
 	}
 
 	var bestAssignment ResourceAssignment
-	bestAssignmentMode := granularMode{preemptionMode: noFit, needsBorrowing: true}
+	bestAssignmentMode := worstGranularMode()
 
 	// We will only check against the flavors' labels for the resource.
 	attemptedFlavorIdx := -1
@@ -682,7 +710,7 @@ func (a *FlavorAssigner) findFlavorForPodSets(
 		}
 		assignments := make(ResourceAssignment, len(requests))
 		// Calculate representativeMode for this assignment as the worst mode among all requests.
-		representativeMode := granularMode{preemptionMode: fit, needsBorrowing: false}
+		representativeMode := bestGranularMode()
 		for rName, val := range requests {
 			// Ensure the same resource flavor is used for the workload slice as in the original admitted slice.
 			if features.Enabled(features.ElasticJobsViaWorkloadSlices) && a.replaceWorkloadSlice != nil {
@@ -692,7 +720,7 @@ func (a *FlavorAssigner) findFlavorForPodSets(
 					// Enforce consistent resource flavor assignment between slices.
 					if originalFlavor := preemptWorkloadRequests.Flavors[rName]; originalFlavor != fName {
 						// Flavor mismatch. Skip further checks for this resource.
-						representativeMode = granularMode{preemptionMode: noFit, needsBorrowing: true}
+						representativeMode = worstGranularMode()
 						status.reasons = append(status.reasons, fmt.Sprintf("could not assign %s flavor since the original workload is assigned: %s", fName, originalFlavor))
 						break
 					}
@@ -709,7 +737,7 @@ func (a *FlavorAssigner) findFlavorForPodSets(
 			if s != nil {
 				status.reasons = append(status.reasons, s.reasons...)
 			}
-			mode := granularMode{preemptionMode, borrow > 0}
+			mode := granularMode{preemptionMode, borrowingLevel(borrow)}
 			if isPreferred(representativeMode, mode, a.cq.FlavorFungibility) {
 				representativeMode = mode
 			}
@@ -809,16 +837,16 @@ func shouldTryNextFlavor(representativeMode granularMode, flavorFungibility kueu
 	policyPreempt := flavorFungibility.WhenCanPreempt
 	policyBorrow := flavorFungibility.WhenCanBorrow
 	if representativeMode.isPreemptMode() && policyPreempt == kueue.Preempt {
-		if !representativeMode.needsBorrowing || policyBorrow == kueue.Borrow {
+		if representativeMode.borrowingLevel.optimal() || policyBorrow == kueue.Borrow {
 			return false
 		}
 	}
 
-	if representativeMode.preemptionMode == fit && representativeMode.needsBorrowing && policyBorrow == kueue.Borrow {
+	if representativeMode.preemptionMode == fit && !representativeMode.borrowingLevel.optimal() && policyBorrow == kueue.Borrow {
 		return false
 	}
 
-	if representativeMode.preemptionMode == fit && !representativeMode.needsBorrowing {
+	if representativeMode.preemptionMode == fit && representativeMode.borrowingLevel.optimal() {
 		return false
 	}
 
diff --git a/site/content/en/docs/tasks/_index.md b/site/content/en/docs/tasks/_index.md
index 8487407b6..f2d62f98e 100755
--- a/site/content/en/docs/tasks/_index.md
+++ b/site/content/en/docs/tasks/_index.md
@@ -56,6 +56,7 @@ As a serving user, you can learn how to:
 - [Run a Kueue managed Deployment](run/deployment).
 - [Run a Kueue managed StatefulSet](run/statefulset).
 - [Run a Kueue managed LeaderWorkerSet](run/leaderworkerset).
+- [Run a Kueue managed KubeRay RayService](run/rayservices).
 
 ### Platform developer
 
diff --git a/site/content/en/docs/tasks/run/rayclusters.md b/site/content/en/docs/tasks/run/rayclusters.md
index 7d2e846bb..80444f485 100644
--- a/site/content/en/docs/tasks/run/rayclusters.md
+++ b/site/content/en/docs/tasks/run/rayclusters.md
@@ -2,9 +2,9 @@
 title: "Run A RayCluster"
 linkTitle: "RayClusters"
 date: 2024-08-07
-weight: 6
+weight: 10
 description: >
-  Run a RayCluster on Kueue.
+  Run a RayCluster with Kueue.
 ---
 
 This page shows how to leverage Kueue's scheduling and resource management capabilities when running [RayCluster](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/raycluster-quick-start.html).
@@ -91,6 +91,6 @@ The RayCluster looks like the following:
 You can submit a Ray Job using the [CLI](https://docs.ray.io/en/latest/cluster/running-applications/job-submission/quickstart.html) or log into the Ray Head and execute a job following this [example](https://ray-project.github.io/kuberay/deploy/helm-cluster/#end-to-end-example) with kind cluster.
 
 {{% alert title="Note" color="primary" %}}
-The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.1.1/ray-operator/config/samples/ray-cluster.complete.yaml)
+The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.4.2/ray-operator/config/samples/ray-cluster.complete.yaml)
 and only has the `queue-name` label added and requests updated.
 {{% /alert %}}
\ No newline at end of file
diff --git a/site/content/en/docs/tasks/run/rayjobs.md b/site/content/en/docs/tasks/run/rayjobs.md
index d05e99c34..eb372fe4d 100644
--- a/site/content/en/docs/tasks/run/rayjobs.md
+++ b/site/content/en/docs/tasks/run/rayjobs.md
@@ -2,9 +2,9 @@
 title: "Run A RayJob"
 linkTitle: "RayJobs"
 date: 2024-08-07
-weight: 6
+weight: 10
 description: >
-  Run a Kueue scheduled RayJob.
+  Run a RayJob with Kueue.
 ---
 
 This page shows how to leverage Kueue's scheduling and resource management capabilities when running [KubeRay's](https://github.com/ray-project/kuberay)
@@ -91,6 +91,6 @@ kubectl create -f ray-job-sample.yaml
 ```
 
 {{% alert title="Note" color="primary" %}}
-The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.1.1/ray-operator/config/samples/ray-job.sample.yaml) 
+The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.4.2/ray-operator/config/samples/ray-job.sample.yaml) 
 and only has the `queue-name` label added and requests updated.
 {{% /alert %}}
\ No newline at end of file
diff --git a/site/content/en/docs/tasks/run/rayservices.md b/site/content/en/docs/tasks/run/rayservices.md
new file mode 100644
index 000000000..46967506c
--- /dev/null
+++ b/site/content/en/docs/tasks/run/rayservices.md
@@ -0,0 +1,79 @@
+---
+title: "Run A RayService"
+linkTitle: "RayServices"
+date: 2025-06-30
+weight: 10
+description: >
+  Run a RayService with Kueue.
+---
+
+This page shows how to leverage Kueue's scheduling and resource management capabilities when running [RayService](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayservice-quick-start.html).
+
+Kueue manages the RayService through the RayCluster created for it. Therefore, RayService needs the label of `kueue.x-k8s.io/queue-name: user-queue` and this label is propagated to the relevant RayCluster to trigger Kueue's management.
+
+This guide is for [serving users](/docs/tasks#serving-user) that have a basic understanding of Kueue. For more information, see [Kueue's overview](/docs/overview).
+
+## Before you begin
+
+1. Make sure you are using Kueue v0.6.0 version or newer and KubeRay v1.3.0 or newer.
+
+2. Check [Administer cluster quotas](/docs/tasks/manage/administer_cluster_quotas) for details on the initial Kueue setup.
+
+3. See [KubeRay Installation](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/kuberay-operator-installation.html) for installation and configuration details of KubeRay.
+
+{{% alert title="Note" color="primary" %}}
+RayService is managed by Kueue through RayCluster, and in order to use RayCluster, prior to v0.8.1, you need to restart Kueue after the installation.
+You can do it by running: `kubectl delete pods -l control-plane=controller-manager -n kueue-system`.
+{{% /alert %}}
+
+## RayService definition
+
+When running [RayService](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayservice-quick-start.html) on Kueue, take into consideration the following aspects:
+
+### a. Queue selection
+
+The target [local queue](/docs/concepts/local_queue) should be specified in the `metadata.labels` section of the RayService configuration, and this label will be propagated to its RayCluster.
+
+```yaml
+metadata:
+  labels:
+    kueue.x-k8s.io/queue-name: user-queue
+```
+
+### b. Configure the resource needs
+
+The resource needs of the workload can be configured in the `spec.rayClusterConfig`.
+
+```yaml
+spec:
+  rayClusterConfig:
+    headGroupSpec:
+    template:
+      spec:
+        containers:
+          - resources:
+              requests:
+                cpu: "1"
+    workerGroupSpecs:
+    - template:
+        spec:
+          containers:
+            - resources:
+                requests:
+                  cpu: "1"
+```
+
+### c. Limitations
+- Limited Worker Groups: Because a Kueue workload can have a maximum of 8 PodSets, the maximum number of `spec.rayClusterConfig.workerGroupSpecs` is 7
+- In-Tree Autoscaling Disabled: Kueue manages resource allocation for the RayService; therefore, the internal autoscaling mechanisms need to be disabled
+
+## Example RayService
+
+The RayService looks like the following:
+
+{{< include "examples/jobs/ray-service-sample.yaml" "yaml" >}}
+
+{{% alert title="Note" color="primary" %}}
+The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.4.2/ray-operator/config/samples/ray-service.sample.yaml)
+and only has the `queue-name` label added and requests updated.
+{{% /alert %}}
\ No newline at end of file
diff --git a/site/content/zh-CN/docs/tasks/_index.md b/site/content/zh-CN/docs/tasks/_index.md
index 973b29c92..b2ade75a2 100755
--- a/site/content/zh-CN/docs/tasks/_index.md
+++ b/site/content/zh-CN/docs/tasks/_index.md
@@ -55,6 +55,7 @@ As a serving user, you can learn how to:
 - [Run a Kueue managed Deployment](run/deployment).
 - [Run a Kueue managed StatefulSet](run/statefulset).
 - [Run a Kueue managed LeaderWorkerSet](run/leaderworkerset).
+- [Run a Kueue managed KubeRay RayService](run/rayservices).
 
 ### Platform developer
 
diff --git a/site/content/zh-CN/docs/tasks/run/rayclusters.md b/site/content/zh-CN/docs/tasks/run/rayclusters.md
index 1a9713e5e..079daa57b 100644
--- a/site/content/zh-CN/docs/tasks/run/rayclusters.md
+++ b/site/content/zh-CN/docs/tasks/run/rayclusters.md
@@ -77,6 +77,6 @@ RayCluster 如下所示：
 提交 Ray Job，或者登录 Ray Head 并按照此 [示例](https://ray-project.github.io/kuberay/deploy/helm-cluster/#end-to-end-example)在 kind 集群中执行作业。
 
 {{% alert title="注意" color="primary" %}}
-上述示例来自 [这里](https://raw.githubusercontent.com/ray-project/kuberay/v1.1.1/ray-operator/config/samples/ray-cluster.complete.yaml)，
+上述示例来自 [这里](https://raw.githubusercontent.com/ray-project/kuberay/v1.4.2/ray-operator/config/samples/ray-cluster.complete.yaml)，
 仅添加了 `queue-name` 标签并更新了请求。
 {{% /alert %}}
\ No newline at end of file
diff --git a/site/content/zh-CN/docs/tasks/run/rayjobs.md b/site/content/zh-CN/docs/tasks/run/rayjobs.md
index 915a85f9e..2f2c35449 100644
--- a/site/content/zh-CN/docs/tasks/run/rayjobs.md
+++ b/site/content/zh-CN/docs/tasks/run/rayjobs.md
@@ -3,7 +3,7 @@ title: "运行 RayJob"
 linkTitle: "RayJobs"
 date: 2024-08-07
 weight: 6
-description: 在启用了 Kueue 的环境里运行 RayJob
+description: 在启用了 Kueue 的环境里运行 RayJobs
 ---
 
 本页面展示了如何利用 Kueue 的调度和服务管理能力来运行 [KubeRay](https://github.com/ray-project/kuberay)
@@ -89,6 +89,6 @@ kubectl create -f ray-job-sample.yaml
 ```
 
 {{% alert title="注意" color="primary" %}}
-上述示例来自[这里](https://raw.githubusercontent.com/ray-project/kuberay/v1.1.1/ray-operator/config/samples/ray-job.sample.yaml)
+上述示例来自[这里](https://raw.githubusercontent.com/ray-project/kuberay/v1.4.2/ray-operator/config/samples/ray-job.sample.yaml)
 并且只添加了 `queue-name` 标签和更新了请求。
 {{% /alert %}}
\ No newline at end of file
diff --git a/site/content/en/docs/tasks/run/rayservice.md b/site/content/zh-CN/docs/tasks/run/rayservices.md
similarity index 76%
rename from site/content/en/docs/tasks/run/rayservice.md
rename to site/content/zh-CN/docs/tasks/run/rayservices.md
index 8240f3315..7feda5830 100644
--- a/site/content/en/docs/tasks/run/rayservice.md
+++ b/site/content/zh-CN/docs/tasks/run/rayservices.md
@@ -4,14 +4,14 @@ linkTitle: "RayServices"
 date: 2025-06-30
 weight: 6
 description: >
-  Run a RayService on Kueue.
+  Run a RayService with Kueue.
 ---
 
 This page shows how to leverage Kueue's scheduling and resource management capabilities when running [RayService](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayservice-quick-start.html).
 
-Kueue manages the RayService through the RayCluster created for it. Therefore, RayService needs the label of kueue.x-k8s.io/queue-name: user-queue and this label is propagated to the relevant RayCluster to trigger Kueue's management.
+Kueue manages the RayService through the RayCluster created for it. Therefore, RayService needs the label of `kueue.x-k8s.io/queue-name: user-queue` and this label is propagated to the relevant RayCluster to trigger Kueue's management.
 
-This guide is for [batch users](/docs/tasks#batch-user) that have a basic understanding of Kueue. For more information, see [Kueue's overview](/docs/overview).
+This guide is for [serving users](/docs/tasks#serving-user) that have a basic understanding of Kueue. For more information, see [Kueue's overview](/docs/overview).
 
 ## Before you begin
 
@@ -19,7 +19,7 @@ This guide is for [batch users](/docs/tasks#batch-user) that have a basic unders
 
 2. Check [Administer cluster quotas](/docs/tasks/manage/administer_cluster_quotas) for details on the initial Kueue setup.
 
-3. See [KubeRay Installation](https://raw.githubusercontent.com/ray-project/kuberay/v1.3.0/ray-operator/config/samples/ray-service.sample.yaml) for installation and configuration details of KubeRay.
+3. See [KubeRay Installation](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/kuberay-operator-installation.html) for installation and configuration details of KubeRay.
 
 {{% alert title="Note" color="primary" %}}
 RayService is managed by Kueue through RayCluster, and in order to use RayCluster, prior to v0.8.1, you need to restart Kueue after the installation.
@@ -28,8 +28,7 @@ You can do it by running: `kubectl delete pods -l control-plane=controller-manag
 
 ## RayService definition
 
-When running [RayService](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayservice-quick-start.html) on
-Kueue, take into consideration the following aspects:
+When running [RayService](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayservice-quick-start.html) on Kueue, take into consideration the following aspects:
 
 ### a. Queue selection
 
@@ -75,6 +74,6 @@ The RayService looks like the following:
 {{< include "examples/jobs/ray-service-sample.yaml" "yaml" >}}
 
 {{% alert title="Note" color="primary" %}}
-The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.3.0/ray-operator/config/samples/ray-service.sample.yaml)
+The example above comes from [here](https://raw.githubusercontent.com/ray-project/kuberay/v1.4.2/ray-operator/config/samples/ray-service.sample.yaml)
 and only has the `queue-name` label added and requests updated.
 {{% /alert %}}
\ No newline at end of file
diff --git a/site/static/_redirects b/site/static/_redirects
index 60eb7a606..128cea0a0 100644
--- a/site/static/_redirects
+++ b/site/static/_redirects
@@ -20,11 +20,11 @@
 /docs/tasks/run_kubeflow_jobs      /docs/tasks/run/kubeflow                                                 301
 /docs/tasks/run_plain_pods         /docs/tasks/run/plain_pods                                               301
 /docs/tasks/run_rayclusters        /docs/tasks/run/rayclusters                                              301
-/docs/tasks/run_rayjobs            /docs/tasks/run/rayjobs                                                  301
+/docs/tasks/run_rayjobs            /docs/tasks/run/rayjobs                                                  301                                                  
+/docs/tasks/run_rayservices        /docs/tasks/run/rayservices                                              301                                                  
 
 /docs/tasks/run_kubeflow_jobs/run_mpijobs     /docs/tasks/run/kubeflow/mpijobs     301
 /docs/tasks/run_kubeflow_jobs/run_paddlejobs  /docs/tasks/run/kubeflow/paddlejobs  301
 /docs/tasks/run_kubeflow_jobs/run_pytorchjobs /docs/tasks/run/kubeflow/pytorchjobs 301
 /docs/tasks/run_kubeflow_jobs/run_tfjobs      /docs/tasks/run/kubeflow/tfjobs      301
 /docs/tasks/run_kubeflow_jobs/run_xgboostjobs /docs/tasks/run/kubeflow/xgboostjobs 301
-



Test patch:
diff --git a/pkg/scheduler/scheduler_test.go b/pkg/scheduler/scheduler_test.go
index 24258dcbc..5651fbb80 100644
--- a/pkg/scheduler/scheduler_test.go
+++ b/pkg/scheduler/scheduler_test.go
@@ -3772,6 +3772,80 @@ func TestSchedule(t *testing.T) {
 				utiltesting.MakeEventRecord("sales", "foo-2", "Admitted", corev1.EventTypeNormal).Obj(),
 			},
 		},
+		"prefer flavor with most local capacity": {
+			enableFairSharing: true,
+			cohorts: []kueue.Cohort{
+				*utiltesting.MakeCohort("root").
+					ResourceGroup(
+						*utiltesting.MakeFlavorQuotas("on-demand").Resource("gpu", "2").Obj(),
+						*utiltesting.MakeFlavorQuotas("spot").Resource("gpu", "0").Obj(),
+					).
+					Obj(),
+				*utiltesting.MakeCohort("cohort-a").
+					Parent("root").
+					ResourceGroup(
+						*utiltesting.MakeFlavorQuotas("on-demand").Resource("gpu", "5").Obj(),
+						*utiltesting.MakeFlavorQuotas("spot").Resource("gpu", "7").Obj(),
+					).
+					Obj(),
+			},
+			additionalClusterQueues: []kueue.ClusterQueue{
+				*utiltesting.MakeClusterQueue("other-alpha").
+					Cohort("cohort-a").
+					Preemption(kueue.ClusterQueuePreemption{
+						WithinClusterQueue:  kueue.PreemptionPolicyLowerPriority,
+						ReclaimWithinCohort: kueue.PreemptionPolicyAny,
+					}).
+					FlavorFungibility(kueue.FlavorFungibility{
+						WhenCanPreempt: kueue.TryNextFlavor,
+						WhenCanBorrow:  kueue.TryNextFlavor,
+					}).
+					ResourceGroup(
+						*utiltesting.MakeFlavorQuotas("on-demand").Resource("gpu", "3").Obj(),
+						*utiltesting.MakeFlavorQuotas("spot").Resource("gpu", "3").Obj(),
+					).
+					Obj(),
+			},
+			additionalLocalQueues: []kueue.LocalQueue{
+				*utiltesting.MakeLocalQueue("other", "eng-alpha").ClusterQueue("other-alpha").Obj(),
+			},
+			workloads: []kueue.Workload{
+				// exhaust quota in on-demand in CohortA
+				*utiltesting.MakeWorkload("a1", "eng-alpha").
+					Queue("other").
+					Request("gpu", "8").
+					SimpleReserveQuota("other-alpha", "on-demand", now).
+					Obj(),
+				// exhaust quota in spot in OtherAlpha CQ
+				*utiltesting.MakeWorkload("a2", "eng-alpha").
+					Queue("other").
+					Request("gpu", "3").
+					SimpleReserveQuota("other-alpha", "spot", now).
+					Obj(),
+				*utiltesting.MakeWorkload("a3", "eng-alpha").
+					Queue("other").
+					Request("gpu", "1").
+					Obj(),
+			},
+			wantScheduled: []workload.Reference{"eng-alpha/a3"},
+			wantAssignments: map[workload.Reference]kueue.Admission{
+				"eng-alpha/a1": *utiltesting.MakeAdmission("other-alpha").
+					PodSets(utiltesting.MakePodSetAssignment(kueue.DefaultPodSetName).
+						Assignment("gpu", "on-demand", "8").
+						Obj()).
+					Obj(),
+				"eng-alpha/a2": *utiltesting.MakeAdmission("other-alpha").
+					PodSets(utiltesting.MakePodSetAssignment(kueue.DefaultPodSetName).
+						Assignment("gpu", "spot", "3").
+						Obj()).
+					Obj(),
+				"eng-alpha/a3": *utiltesting.MakeAdmission("other-alpha").
+					PodSets(utiltesting.MakePodSetAssignment(kueue.DefaultPodSetName).
+						Assignment("gpu", "spot", "1").
+						Obj()).
+					Obj(),
+			},
+		},
 	}
 	for name, tc := range cases {
 		t.Run(name, func(t *testing.T) {
diff --git a/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go b/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go
index ba28a000d..c45469a2f 100644
--- a/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go
+++ b/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go
@@ -40,6 +40,8 @@ import (
 var _ = ginkgo.Describe("Scheduler", ginkgo.Ordered, ginkgo.ContinueOnFailure, func() {
 	var (
 		defaultFlavor *kueue.ResourceFlavor
+		flavor1       *kueue.ResourceFlavor
+		flavor2       *kueue.ResourceFlavor
 		ns            *corev1.Namespace
 
 		cohorts []*kueue.Cohort
@@ -98,6 +100,10 @@ var _ = ginkgo.Describe("Scheduler", ginkgo.Ordered, ginkgo.ContinueOnFailure, f
 	ginkgo.BeforeEach(func() {
 		defaultFlavor = testing.MakeResourceFlavor("default").Obj()
 		util.MustCreate(ctx, k8sClient, defaultFlavor)
+		flavor1 = testing.MakeResourceFlavor("flavor1").Obj()
+		util.MustCreate(ctx, k8sClient, flavor1)
+		flavor2 = testing.MakeResourceFlavor("flavor2").Obj()
+		util.MustCreate(ctx, k8sClient, flavor2)
 
 		ns = util.CreateNamespaceFromPrefixWithLog(ctx, k8sClient, "core-")
 	})
@@ -117,6 +123,8 @@ var _ = ginkgo.Describe("Scheduler", ginkgo.Ordered, ginkgo.ContinueOnFailure, f
 			util.ExpectObjectToBeDeleted(ctx, k8sClient, cohort, true)
 		}
 		util.ExpectObjectToBeDeleted(ctx, k8sClient, defaultFlavor, true)
+		util.ExpectObjectToBeDeleted(ctx, k8sClient, flavor1, true)
+		util.ExpectObjectToBeDeleted(ctx, k8sClient, flavor2, true)
 	})
 
 	ginkgo.When("Preemption is disabled", func() {
@@ -482,6 +490,132 @@ var _ = ginkgo.Describe("Scheduler", ginkgo.Ordered, ginkgo.ContinueOnFailure, f
 		})
 	})
 
+	ginkgo.When("using hierarchical cohorts with several flavors", func() {
+		var (
+			cqp1 *kueue.ClusterQueue
+		)
+		ginkgo.BeforeEach(func() {
+			createCohort(testing.MakeCohort("root-cohort").Obj())
+			createCohort(testing.MakeCohort("cohort-a").
+				Parent("root-cohort").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+				).
+				Obj())
+			createCohort(testing.MakeCohort("cohort-b").
+				Parent("root-cohort").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+				).
+				Obj())
+
+			fungibility := kueue.FlavorFungibility{
+				WhenCanBorrow:  kueue.TryNextFlavor,
+				WhenCanPreempt: kueue.TryNextFlavor,
+			}
+			preemption := kueue.ClusterQueuePreemption{
+				ReclaimWithinCohort: kueue.PreemptionPolicyAny,
+				WithinClusterQueue:  kueue.PreemptionPolicyLowerPriority,
+			}
+
+			cqp1 = createQueue(testing.MakeClusterQueue("cq-p1").
+				Cohort("cohort-a").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "3", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "3", "10").Obj(),
+				).
+				FlavorFungibility(fungibility).
+				Preemption(preemption).
+				Obj())
+
+			createQueue(testing.MakeClusterQueue("cq-p2").
+				Cohort("cohort-a").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "3", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "3", "10").Obj(),
+				).
+				FlavorFungibility(fungibility).
+				Preemption(preemption).
+				Obj())
+
+			createQueue(testing.MakeClusterQueue("cq-p3").
+				Cohort("cohort-a").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "2", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "4", "10").Obj(),
+				).
+				FlavorFungibility(fungibility).
+				Preemption(preemption).
+				Obj())
+
+			createQueue(testing.MakeClusterQueue("cq-p4").
+				Cohort("cohort-b").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "2", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+				).
+				FlavorFungibility(fungibility).
+				Preemption(preemption).
+				Obj())
+
+			createQueue(testing.MakeClusterQueue("cq-p5").
+				Cohort("cohort-b").
+				FairWeight(resource.MustParse("1")).
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("flavor1").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+					*testing.MakeFlavorQuotas("flavor2").Resource(corev1.ResourceCPU, "0", "10").Obj(),
+				).
+				FlavorFungibility(fungibility).
+				Preemption(preemption).
+				Obj())
+		})
+		ginkgo.AfterEach(func() {
+			_ = features.SetEnable(features.FlavorFungibilityImplicitPreferenceDefault, false)
+		})
+
+		// Since CohortA has 18CPU quota, we expect that
+		// all 18 CPUs scheduled are within this guarantee.
+		// On the otherhand, cq-p1 only has 6 quota, and
+		// has to borrow 12. There are 20 cpus are available from
+		// perspective of CohortA, and root, which serve as denominators.
+		//
+		// WeightedShare(CohortA) = 0/20 * 1000 = 0
+		// WeightedShare(cq-p1)  = 12/20 * 1000 = 600
+		ginkgo.It("Prefers flavor with remaining guarantees at Cohort level (FlavorFungibilityImplicitPreferenceDefault=false)", func() {
+			_ = features.SetEnable(features.FlavorFungibilityImplicitPreferenceDefault, false)
+			ginkgo.By("Creating workloads")
+			for range 18 {
+				createWorkload("cq-p1", "1")
+			}
+			ginkgo.By("Workloads active")
+			util.ExpectReservingActiveWorkloadsMetric(cqp1, 18)
+
+			ginkgo.By("Expected Weighted Shares")
+			util.ExpectClusterQueueWeightedShareMetric(cqp1, 600)
+			expectCohortWeightedShare("cohort-a", 0)
+		})
+		ginkgo.It("Prefers flavor with remaining guarantees at Cohort level (FlavorFungibilityImplicitPreferenceDefault=true)", func() {
+			_ = features.SetEnable(features.FlavorFungibilityImplicitPreferenceDefault, true)
+			for range 18 {
+				createWorkload("cq-p1", "1")
+			}
+			ginkgo.By("Workloads active")
+			util.ExpectReservingActiveWorkloadsMetric(cqp1, 18)
+
+			ginkgo.By("Expected Weighted Shares")
+			util.ExpectClusterQueueWeightedShareMetric(cqp1, 600)
+			expectCohortWeightedShare("cohort-a", 0)
+		})
+	})
+
 	ginkgo.When("Using AdmissionFairSharing at ClusterQueue level", func() {
 		var (
 			cq1 *kueue.ClusterQueue