Repository: ray-project/ray
Pull request #51095: [Doc][KubeRay] Add a doc to explain why some worker Pods are not ready in RayService
Author: login: kenchung285
Labels: name: go
description: add ONLY when ready to merge, run all tests
name: community-backlog
Created: 2025-03-05T08:36:32Z | Merged: 2025-03-22T05:16:29Z


## Why are these changes needed?

A worker pod in RayService without a ray serve replica will be set as readiness probe failed.
We add an YAML and example python file in ray-project/kuberay, and add an document in Ray Document.

## Related issue number

[kuberay-#3048](https://github.com/ray-project/kuberay/issues/3048)

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for https://docs.ray.io/en/master/.
    - [ ] I've added any new APIs to the API Reference. For example, if I added a
           method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(



Diff:
diff --git a/doc/source/cluster/kubernetes/images/rayservice-no-ray-serve-replica-dashboard.png b/doc/source/cluster/kubernetes/images/rayservice-no-ray-serve-replica-dashboard.png
new file mode 100644
index 000000000000..2f97a7111d2a
Binary files /dev/null and b/doc/source/cluster/kubernetes/images/rayservice-no-ray-serve-replica-dashboard.png differ
diff --git a/doc/source/cluster/kubernetes/user-guides.md b/doc/source/cluster/kubernetes/user-guides.md
index 2fcc503c7a7e..c8e6fa187655 100644
--- a/doc/source/cluster/kubernetes/user-guides.md
+++ b/doc/source/cluster/kubernetes/user-guides.md
@@ -6,6 +6,7 @@
 :hidden:
 
 Deploy Ray Serve Apps <user-guides/rayservice>
+user-guides/rayservice-no-ray-serve-replica
 user-guides/rayservice-high-availability
 user-guides/observability
 user-guides/upgrade-guide
@@ -38,6 +39,7 @@ at the {ref}`introductory guide <kuberay-quickstart>` first.
 :::
 
 * {ref}`kuberay-rayservice`
+* {ref}`kuberay-rayservice-no-ray-serve-replica`
 * {ref}`kuberay-rayservice-ha`
 * {ref}`kuberay-observability`
 * {ref}`kuberay-upgrade-guide`
diff --git a/doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md b/doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
new file mode 100755
index 000000000000..4db133c65020
--- /dev/null
+++ b/doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
@@ -0,0 +1,194 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService worker Pods aren't ready
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing ProxyActor from sending requests to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+To illustrate, the following example serves one simple Ray Serve app using RayService.
+
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas, that handle requests to this deployment, to run. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on a single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod.
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+KubeRay sets up the worker Pod with a Ray Serve replica with a Proxy actor and marks it as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve apps
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+Note that a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` are running on one of the worker pod, while no Ray Serve replica and Proxy actor is running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve apps by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers that have Ray Serve replicas.
+Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with a Ray Serve replica running. Therefore, users can still send requests to the app and receive responses from it.
+
+```sh
+# Step 6.1: Run a curl Pod.
+# If you already have a curl Pod, you can use `kubectl exec -it <curl-pod> -- sh` to access the Pod.
+kubectl run curl --image=radial/busyboxplus:curl -i --tty
+
+# Step 6.2: Send a request to the simple_app.
+curl -X POST -H 'Content-Type: application/json' rayservice-no-ray-serve-replica-serve-svc:8000/basic
+# [Expected output]: hello world
+```
+
+## Step 7: In-place update for Ray Serve apps
+
+Update the `num_replicas` for the app from `1` to `2` in `ray-service.no-ray-serve-replica.yaml`. This change reconfigures the existing RayCluster.
+
+```sh
+# Step 7.1: Update the num_replicas of the app from 1 to 2.
+# [ray-service.no-ray-serve-replica.yaml]
+# deployments:
+#   - name: BaseService
+#     num_replicas: 2
+#     max_replicas_per_node: 1
+#     ray_actor_options:
+#       num_cpus: 0.1
+
+# Step 7.2: Apply the updated RayService config.
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+
+# Step 7.3: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          46m
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          46m
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   1/1     Running   0          46m
+```
+
+After reconfiguration, KubeRay requests the head Pod to create an additional Ray Serve replica to match the `num_replicas` configuration. Because the `max_replicas_per_node` is `1`, the new Ray Serve replica runs on the worker Pod without any replicas. After that, KubeRay marks the worker Pod as ready.
+
+## Step 8: Clean up the Kubernetes cluster
+
+```sh
+# Delete the RayService.
+kubectl delete -f ray-service.no-ray-serve-replica.yaml
+
+# Uninstall the KubeRay operator.
+helm uninstall kuberay-operator
+
+# Delete the curl Pod.
+kubectl delete pod curl
+```
+
+## Next steps
+
+* See [RayService troubleshooting guide](kuberay-raysvc-troubleshoot) if you encounter any issues.
+* See [Examples](kuberay-examples) for more RayService examples.


Reviews:
user: login: rueian
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T00:50:21Z
user: login: MortalHappiness
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-11T03:36:25Z
user: login: rueian
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T04:35:12Z
user: login: kenchung285
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T04:47:32Z
user: login: rueian
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T05:03:18Z
user: login: rueian
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T15:49:17Z
user: login: rueian
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T16:06:38Z
user: login: kevin85421
body: Have you installed Vale?
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-14T23:31:13Z
user: login: kenchung285
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T01:55:46Z
user: login: kenchung285
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T01:58:02Z
user: login: kenchung285
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T02:09:32Z
user: login: kevin85421
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-17T06:11:48Z
user: login: rueian
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T18:52:08Z
user: login: MortalHappiness
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-18T03:35:24Z
user: login: angelinalg
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T16:58:35Z

Review comments:
diff_hunk: @@ -0,0 +1,220 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+In this guide, we will explore a specific scenario in KubeRay v1.3.0 with Ray 2.41.0 where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: rueian
body: ```suggestion
In this guide, we will explore a specific scenario in KubeRay v1.3.0 with Ray 2.41.0 where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
```
Does the new Ray or new KubeRay still have the same behavior?
created_at: 2025-03-11T00:50:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,220 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+In this guide, we will explore a specific scenario in KubeRay v1.3.0 with Ray 2.41.0 where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## Prerequisites
+
+This guide mainly focuses on the behavior of KubeRay v1.3.0 and Ray 2.41.0.
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. The number of replicas can be scaled based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/v1.3.0/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: rueian
body: 
<img width="971" alt="image" src="https://github.com/user-attachments/assets/9e1dee1a-a7f9-4ff6-b14c-c0fcc754609f" />


created_at: 2025-03-11T04:35:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,220 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+In this guide, we will explore a specific scenario in KubeRay v1.3.0 with Ray 2.41.0 where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kenchung285
body: Yes it does. Kuberay operator sets `readiness probe failed` to the worker pod when no Ray Serve replica exits in a worker pod.
created_at: 2025-03-11T04:47:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,220 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+In this guide, we will explore a specific scenario in KubeRay v1.3.0 with Ray 2.41.0 where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: rueian
body: Should we simply remove those versions? The current statement appears to indicate that the behavior doesn’t apply to later versions.
created_at: 2025-03-11T05:03:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,220 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## Prerequisites
+
+This guide mainly focuses on the behavior of KubeRay v1.3.0 and Ray 2.41.0.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: rueian
body: ```suggestion
```
created_at: 2025-03-11T15:49:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: ```suggestion
# Why are some RayService worker Pods not ready?
```
created_at: 2025-03-14T22:49:10Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: ```suggestion
This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
```
created_at: 2025-03-14T22:55:36Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Please remove the entire section titled `What's a Ray Serve Replica`?. We want to minimize the explanation of Ray Serve concepts in the KubeRay docs. Instead, we should include links to the Ray Serve docs.

```
To better understand this section, you should be familiar with the following Ray Serve components: 
the [Ray Serve replica](link1) and the [ProxyActor](link2).

ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.

The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
See the following example for more details.
```

I don't actually spend too much time polishing the above sentences. Please polish by yourself and make sure it follows Vale rules.
created_at: 2025-03-14T22:59:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: I prefer not to maintain two tabs for every example. We can use x86-64 one directly.
created_at: 2025-03-14T23:14:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Does it have two Ray Serve applications?
created_at: 2025-03-14T23:15:28Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: ```suggestion
  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
```
created_at: 2025-03-14T23:17:52Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Delete this section.
created_at: 2025-03-14T23:23:14Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
+
+```sh
+# Step 4.1: List all RayService custom resources in the `default` namespace.
+kubectl get rayservice
+
+# [Example output]
+# NAME                              SERVICE STATUS   NUM SERVE ENDPOINTS
+# rayservice-no-ray-serve-replica   Running          2
+
+# Step 4.2: List all RayCluster custom resources in the `default` namespace.
+kubectl get raycluster
+
+# [Example output]
+# NAME                                               DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28   2                 2                   1      6Gi      0               87s
+
+# Step 4.3: List services in the `default` namespace.
+kubectl get services
+
+# [Example output]
+# NAME                                                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                         AGE
+# ...
+# ice-no-ray-serve-replica-raycluster-dnm28-head-svc   ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m22s
+# rayservice-no-ray-serve-replica-head-svc             ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m1s
+# rayservice-no-ray-serve-replica-serve-svc            ClusterIP   10.96.8.250   <none>        8000/TCP                                        3m1s
+```
+
+## Step 5: Why 1 worker Pod isn't ready?
+
+```sh
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Add something similar to Step 4.1 in https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/rayservice-high-availability.html#step-4-verify-the-kubernetes-serve-service
created_at: 2025-03-14T23:25:26Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
+
+```sh
+# Step 4.1: List all RayService custom resources in the `default` namespace.
+kubectl get rayservice
+
+# [Example output]
+# NAME                              SERVICE STATUS   NUM SERVE ENDPOINTS
+# rayservice-no-ray-serve-replica   Running          2
+
+# Step 4.2: List all RayCluster custom resources in the `default` namespace.
+kubectl get raycluster
+
+# [Example output]
+# NAME                                               DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28   2                 2                   1      6Gi      0               87s
+
+# Step 4.3: List services in the `default` namespace.
+kubectl get services
+
+# [Example output]
+# NAME                                                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                         AGE
+# ...
+# ice-no-ray-serve-replica-raycluster-dnm28-head-svc   ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m22s
+# rayservice-no-ray-serve-replica-head-svc             ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m1s
+# rayservice-no-ray-serve-replica-serve-svc            ClusterIP   10.96.8.250   <none>        8000/TCP                                        3m1s
+```
+
+## Step 5: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 5.1: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 5.2: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+KubeRay creates a RayCluster based on `spec.rayClusterConfig` defined in the RayService YAML for a RayService custom resource.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Delete L131 - L133
created_at: 2025-03-14T23:26:31Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
+
+```sh
+# Step 4.1: List all RayService custom resources in the `default` namespace.
+kubectl get rayservice
+
+# [Example output]
+# NAME                              SERVICE STATUS   NUM SERVE ENDPOINTS
+# rayservice-no-ray-serve-replica   Running          2
+
+# Step 4.2: List all RayCluster custom resources in the `default` namespace.
+kubectl get raycluster
+
+# [Example output]
+# NAME                                               DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28   2                 2                   1      6Gi      0               87s
+
+# Step 4.3: List services in the `default` namespace.
+kubectl get services
+
+# [Example output]
+# NAME                                                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                         AGE
+# ...
+# ice-no-ray-serve-replica-raycluster-dnm28-head-svc   ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m22s
+# rayservice-no-ray-serve-replica-head-svc             ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m1s
+# rayservice-no-ray-serve-replica-serve-svc            ClusterIP   10.96.8.250   <none>        8000/TCP                                        3m1s
+```
+
+## Step 5: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 5.1: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 5.2: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+KubeRay creates a RayCluster based on `spec.rayClusterConfig` defined in the RayService YAML for a RayService custom resource.  
+Next, once the head Pod is running and ready, KubeRay submits a request to the head's dashboard port to create the Ray Serve applications defined in `spec.serveConfigV2`.  
+KubeRay deploys these Ray Serve applications as Ray Serve replicas in worker Pods.  
+
+Look at the output of Step 4.3. One worker Pod is running and ready, while the other is running but not ready.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Step 4.3?
created_at: 2025-03-14T23:28:30Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
+
+```sh
+# Step 4.1: List all RayService custom resources in the `default` namespace.
+kubectl get rayservice
+
+# [Example output]
+# NAME                              SERVICE STATUS   NUM SERVE ENDPOINTS
+# rayservice-no-ray-serve-replica   Running          2
+
+# Step 4.2: List all RayCluster custom resources in the `default` namespace.
+kubectl get raycluster
+
+# [Example output]
+# NAME                                               DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28   2                 2                   1      6Gi      0               87s
+
+# Step 4.3: List services in the `default` namespace.
+kubectl get services
+
+# [Example output]
+# NAME                                                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                         AGE
+# ...
+# ice-no-ray-serve-replica-raycluster-dnm28-head-svc   ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m22s
+# rayservice-no-ray-serve-replica-head-svc             ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m1s
+# rayservice-no-ray-serve-replica-serve-svc            ClusterIP   10.96.8.250   <none>        8000/TCP                                        3m1s
+```
+
+## Step 5: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 5.1: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 5.2: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+KubeRay creates a RayCluster based on `spec.rayClusterConfig` defined in the RayService YAML for a RayService custom resource.  
+Next, once the head Pod is running and ready, KubeRay submits a request to the head's dashboard port to create the Ray Serve applications defined in `spec.serveConfigV2`.  
+KubeRay deploys these Ray Serve applications as Ray Serve replicas in worker Pods.  
+
+Look at the output of Step 4.3. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 6: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 7: Send requests to the Serve applications by the Kubernetes serve service
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kevin85421
body: Delete Step 7 which is out of scope of this doc.
created_at: 2025-03-14T23:30:39Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
+
+```sh
+# Step 4.1: List all RayService custom resources in the `default` namespace.
+kubectl get rayservice
+
+# [Example output]
+# NAME                              SERVICE STATUS   NUM SERVE ENDPOINTS
+# rayservice-no-ray-serve-replica   Running          2
+
+# Step 4.2: List all RayCluster custom resources in the `default` namespace.
+kubectl get raycluster
+
+# [Example output]
+# NAME                                               DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28   2                 2                   1      6Gi      0               87s
+
+# Step 4.3: List services in the `default` namespace.
+kubectl get services
+
+# [Example output]
+# NAME                                                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                         AGE
+# ...
+# ice-no-ray-serve-replica-raycluster-dnm28-head-svc   ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m22s
+# rayservice-no-ray-serve-replica-head-svc             ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m1s
+# rayservice-no-ray-serve-replica-serve-svc            ClusterIP   10.96.8.250   <none>        8000/TCP                                        3m1s
+```
+
+## Step 5: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 5.1: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 5.2: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+KubeRay creates a RayCluster based on `spec.rayClusterConfig` defined in the RayService YAML for a RayService custom resource.  
+Next, once the head Pod is running and ready, KubeRay submits a request to the head's dashboard port to create the Ray Serve applications defined in `spec.serveConfigV2`.  
+KubeRay deploys these Ray Serve applications as Ray Serve replicas in worker Pods.  
+
+Look at the output of Step 4.3. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 6: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 7: Send requests to the Serve applications by the Kubernetes serve service
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kenchung285
body: The reason I add Step 7 is that I want to demonstrate that the application still works even if one worker pod is marked as unready. I can move the description above the shell command to make the purpose of this step more clearly
created_at: 2025-03-15T01:55:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kenchung285
body: No, it has only one. I'll modify it.
created_at: 2025-03-15T01:58:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,216 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# RayService Worker No Ray Serve Replica
+
+This guide explores a specific scenario in KubeRay with Ray where a Ray worker Pod remains in an unready state due to the lack of a Ray Serve replica.  
+
+## What's a Ray Serve Replica?
+
+A Ray Serve Replica is an instance of a deployment in Ray Serve that processes incoming requests. Each replica runs as a Ray actor and can handle HTTP or Python function calls. Users can scale the number of replicas based on workload requirements, distributing traffic across the available replicas for load balancing and high availability.  
+
+## Example: Serve two simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+::::{tab-set}
+
+:::{tab-item} x86-64 (Intel/Linux)
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+:::{tab-item} ARM64 (Apple silicon)
+```sh
+curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml | sed 's/2.41.0/2.41.0-aarch64/g' > ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+:::
+
+::::
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the ray serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Verify the Kubernetes cluster status
+
+```sh
+# Step 4.1: List all RayService custom resources in the `default` namespace.
+kubectl get rayservice
+
+# [Example output]
+# NAME                              SERVICE STATUS   NUM SERVE ENDPOINTS
+# rayservice-no-ray-serve-replica   Running          2
+
+# Step 4.2: List all RayCluster custom resources in the `default` namespace.
+kubectl get raycluster
+
+# [Example output]
+# NAME                                               DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28   2                 2                   1      6Gi      0               87s
+
+# Step 4.3: List services in the `default` namespace.
+kubectl get services
+
+# [Example output]
+# NAME                                                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                         AGE
+# ...
+# ice-no-ray-serve-replica-raycluster-dnm28-head-svc   ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m22s
+# rayservice-no-ray-serve-replica-head-svc             ClusterIP   None          <none>        10001/TCP,8265/TCP,6379/TCP,8080/TCP,8000/TCP   3m1s
+# rayservice-no-ray-serve-replica-serve-svc            ClusterIP   10.96.8.250   <none>        8000/TCP                                        3m1s
+```
+
+## Step 5: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 5.1: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 5.2: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+KubeRay creates a RayCluster based on `spec.rayClusterConfig` defined in the RayService YAML for a RayService custom resource.  
+Next, once the head Pod is running and ready, KubeRay submits a request to the head's dashboard port to create the Ray Serve applications defined in `spec.serveConfigV2`.  
+KubeRay deploys these Ray Serve applications as Ray Serve replicas in worker Pods.  
+
+Look at the output of Step 4.3. One worker Pod is running and ready, while the other is running but not ready.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: kenchung285
body: It should be 5.1. I copied from other lines but forgot to change the number.
created_at: 2025-03-15T02:09:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
# RayService worker Pods aren't ready
```
created_at: 2025-03-20T16:37:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
KubeRay's readiness probe fails, rendering the Pods unready and preventing ProxyActor from sending requests to them.
```
created_at: 2025-03-20T16:40:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
```
created_at: 2025-03-20T16:40:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
  * `num_replicas`: Controls the number of replicas, that handle requests to this deployment, to run. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
```
created_at: 2025-03-20T16:42:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
  * `max_replicas_per_node`: Controls the maximum number of replicas on a single pod.  
```
created_at: 2025-03-20T16:42:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod.
```
created_at: 2025-03-20T16:43:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
KubeRay sets up the worker Pod with a Ray Serve replica with a Proxy actor and marks it as ready.
```
created_at: 2025-03-20T16:45:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
## Step 5: Verify the status of the Serve apps
```
created_at: 2025-03-20T16:47:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
Note that a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` are running on one of the worker pod, while no Ray Serve replica and Proxy actor is running on the another. KubeRay marks the former as ready and the later as unready.
```
created_at: 2025-03-20T16:48:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
## Step 6: Send requests to the Serve apps by the Kubernetes serve service
```
created_at: 2025-03-20T16:49:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers which have Ray Serve replicas.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers that have Ray Serve replicas.
```
created_at: 2025-03-20T16:49:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers which have Ray Serve replicas.
+Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with Ray Serve replica running. Therefore, users can still send requests to the application and receive responses from it.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with a Ray Serve replica running. Therefore, users can still send requests to the app and receive responses from it.
```
created_at: 2025-03-20T16:50:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers which have Ray Serve replicas.
+Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with Ray Serve replica running. Therefore, users can still send requests to the application and receive responses from it.
+
+```sh
+# Step 6.1: Run a curl Pod.
+# If you already have a curl Pod, you can use `kubectl exec -it <curl-pod> -- sh` to access the Pod.
+kubectl run curl --image=radial/busyboxplus:curl -i --tty
+
+# Step 6.2: Send a request to the simple_app.
+curl -X POST -H 'Content-Type: application/json' rayservice-no-ray-serve-replica-serve-svc:8000/basic
+# [Expected output]: hello world
+```
+
+## Step 7: In-place update for Ray Serve applications
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
## Step 7: In-place update for Ray Serve apps
```
created_at: 2025-03-20T16:50:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers which have Ray Serve replicas.
+Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with Ray Serve replica running. Therefore, users can still send requests to the application and receive responses from it.
+
+```sh
+# Step 6.1: Run a curl Pod.
+# If you already have a curl Pod, you can use `kubectl exec -it <curl-pod> -- sh` to access the Pod.
+kubectl run curl --image=radial/busyboxplus:curl -i --tty
+
+# Step 6.2: Send a request to the simple_app.
+curl -X POST -H 'Content-Type: application/json' rayservice-no-ray-serve-replica-serve-svc:8000/basic
+# [Expected output]: hello world
+```
+
+## Step 7: In-place update for Ray Serve applications
+
+Update the `num_replicas` of application from `1` to `2` in `ray-service.no-ray-serve-replica.yaml`. This change reconfigures the existing RayCluster.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
Update the `num_replicas` for the app from `1` to `2` in `ray-service.no-ray-serve-replica.yaml`. This change reconfigures the existing RayCluster.
```
created_at: 2025-03-20T16:50:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers which have Ray Serve replicas.
+Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with Ray Serve replica running. Therefore, users can still send requests to the application and receive responses from it.
+
+```sh
+# Step 6.1: Run a curl Pod.
+# If you already have a curl Pod, you can use `kubectl exec -it <curl-pod> -- sh` to access the Pod.
+kubectl run curl --image=radial/busyboxplus:curl -i --tty
+
+# Step 6.2: Send a request to the simple_app.
+curl -X POST -H 'Content-Type: application/json' rayservice-no-ray-serve-replica-serve-svc:8000/basic
+# [Expected output]: hello world
+```
+
+## Step 7: In-place update for Ray Serve applications
+
+Update the `num_replicas` of application from `1` to `2` in `ray-service.no-ray-serve-replica.yaml`. This change reconfigures the existing RayCluster.
+
+```sh
+# Step 7.1: Update the num_replicas of application from 1 to 2.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
# Step 7.1: Update the num_replicas of the app from 1 to 2.
```
created_at: 2025-03-20T16:51:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
+
+## Example: Serve one simple Ray Serve applications using RayService
+
+## Step 1: Create a Kubernetes cluster with Kind
+
+```sh
+kind create cluster --image=kindest/node:v1.26.0
+```
+
+## Step 2: Install the KubeRay operator
+
+Follow [this document](kuberay-operator-deploy) to install the latest stable KubeRay operator using Helm repository.
+
+## Step 3: Install a RayService
+
+```sh
+curl -O https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.no-ray-serve-replica.yaml
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+```
+
+Look at the Ray Serve configuration `serveConfigV2` embedded in the RayService YAML. Notice the only deployment in `deployments` of the application named `simple_app`:
+  * `num_replicas`: Controls the number of replicas to run that handle requests to this deployment. Initialize to 1 to ensure the overall number of the Ray Serve replicas is 1.
+  * `max_replicas_per_node`: Controls the maximum number of replicas on one single pod.  
+
+See [Ray Serve Documentation](https://docs.ray.io/en/master/serve/configure-serve-deployment.html) for more details.
+```yaml
+serveConfigV2: |
+  applications:
+    - name: simple_app
+      import_path: ray-operator.config.samples.ray-serve.single_deployment_dag:DagNode
+      route_prefix: /basic
+      runtime_env:
+        working_dir: "https://github.com/ray-project/kuberay/archive/master.zip"
+      deployments:
+        - name: BaseService
+          num_replicas: 1
+          max_replicas_per_node: 1
+          ray_actor_options:
+            num_cpus: 0.1
+```
+
+Look at the head Pod configuration `rayClusterConfig:headGroupSpec` embedded in the RayService YAML.  
+The configuration sets the CPU resources for the head Pod to 0 by passing the option `num-cpus: "0"` to `rayStartParams`. This setup avoids Ray Serve replicas running on the head Pod
+See [rayStartParams](https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md) for more details.  
+```sh
+headGroupSpec:
+  rayStartParams:
+    num-cpus: "0"
+  template: ...
+```
+
+## Step 4: Why 1 worker Pod isn't ready?
+
+```sh
+# Step 4.1: Wait until the RayService is ready to serve requests.
+kubectl describe rayservices.ray.io rayservice-no-ray-serve-replica
+
+# [Example output]
+#  Conditions:
+#    Last Transition Time:  2025-03-18T14:14:43Z
+#    Message:               Number of serve endpoints is greater than 0
+#    Observed Generation:   1
+#    Reason:                NonZeroServeEndpoints
+#    Status:                True
+#    Type:                  Ready
+#    Last Transition Time:  2025-03-18T14:12:03Z
+#    Message:               Active Ray cluster exists and no pending Ray cluster
+#    Observed Generation:   1
+#    Reason:                NoPendingCluster
+#    Status:                False
+#    Type:                  UpgradeInProgress
+
+# Step 4.2: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          2m21s
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   0/1     Running   0          2m20s
+
+# Step 4.3: Check unready worker pod events
+kubectl describe pods {YOUR_UNREADY_WORKER_POD_NAME}
+
+# [Example output]
+# Events:
+#   Type     Reason     Age                   From               Message
+#   ----     ------     ----                  ----               -------
+#   Normal   Scheduled  3m4s                  default-scheduler  Successfully assigned default/rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk to kind-control-plane
+#   Normal   Pulled     3m3s                  kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    3m3s                  kubelet            Created container wait-gcs-ready
+#   Normal   Started    3m3s                  kubelet            Started container wait-gcs-ready
+#   Normal   Pulled     2m57s                 kubelet            Container image "rayproject/ray:2.41.0" already present on machine
+#   Normal   Created    2m57s                 kubelet            Created container ray-worker
+#   Normal   Started    2m57s                 kubelet            Started container ray-worker
+#   Warning  Unhealthy  78s (x19 over 2m43s)  kubelet            Readiness probe failed: success
+```
+
+Look at the output of Step 4.2. One worker Pod is running and ready, while the other is running but not ready.  
+Starting from Ray 2.8, a Ray worker Pod that doesn't have any Ray Serve replica won't have a Proxy actor.  
+Starting from KubeRay v1.1.0, KubeRay adds a readiness probe to every worker Pod's Ray container to check if the worker Pod has a Proxy actor or not.  
+If the worker Pod lacks a Proxy actor, the readiness probe fails, rendering the worker Pod unready, and thus, it doesn't receive any traffic.  
+
+With `spec.serveConfigV2`, KubeRay only creates one Ray Serve replica and schedules it to one of the worker Pods.
+The worker Pod with a Ray Serve replica is setup with a Proxy actor and marked as ready.
+KubeRay marks the other worker Pod, which doesn't have any Ray Serve replica and a Proxy actor, as unready.
+
+## Step 5: Verify the status of the Serve applications
+
+```sh
+kubectl port-forward svc/rayservice-no-ray-serve-replica-head-svc 8265:8265
+```
+
+See [rayservice-troubleshooting.md](kuberay-raysvc-troubleshoot) for more details on RayService observability.  
+
+Below is a screenshot example of the Serve page in the Ray dashboard.  
+There's a `ray::ServeReplica::simple_app::BaseService` and a `ray::ProxyActor` running on one of the worker pod, while no Ray Serve replica and Proxy actor running on the another. KubeRay marks the former as ready and the later as unready.
+  ![Ray Serve Dashboard](../images/rayservice-no-ray-serve-replica-dashboard.png)
+
+## Step 6: Send requests to the Serve applications by the Kubernetes serve service
+
+`rayservice-no-ray-serve-serve-svc` does traffic routing among all the workers which have Ray Serve replicas.
+Although one worker Pod is unready, Ray Serve can still route the traffic to the ready worker Pod with Ray Serve replica running. Therefore, users can still send requests to the application and receive responses from it.
+
+```sh
+# Step 6.1: Run a curl Pod.
+# If you already have a curl Pod, you can use `kubectl exec -it <curl-pod> -- sh` to access the Pod.
+kubectl run curl --image=radial/busyboxplus:curl -i --tty
+
+# Step 6.2: Send a request to the simple_app.
+curl -X POST -H 'Content-Type: application/json' rayservice-no-ray-serve-replica-serve-svc:8000/basic
+# [Expected output]: hello world
+```
+
+## Step 7: In-place update for Ray Serve applications
+
+Update the `num_replicas` of application from `1` to `2` in `ray-service.no-ray-serve-replica.yaml`. This change reconfigures the existing RayCluster.
+
+```sh
+# Step 7.1: Update the num_replicas of application from 1 to 2.
+# [ray-service.no-ray-serve-replica.yaml]
+# deployments:
+#   - name: BaseService
+#     num_replicas: 2
+#     max_replicas_per_node: 1
+#     ray_actor_options:
+#       num_cpus: 0.1
+
+# Step 7.2: Apply the updated RayService config.
+kubectl apply -f ray-service.no-ray-serve-replica.yaml
+
+# Step 7.3: List all Ray Pods in the `default` namespace.
+kubectl get pods -l=ray.io/is-ray-node=yes
+
+# [Example output]
+# NAME                                                              READY   STATUS    RESTARTS   AGE
+# rayservice-no-ray-serve-replica-raycluster-dnm28-head-9h2qt       1/1     Running   0          46m
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-46t7l   1/1     Running   0          46m
+# rayservice-no-ray-serve-replica-raycluster-dnm28-s-worker-77rzk   1/1     Running   0          46m
+```
+
+After reconfiguration, KubeRay requests the head Pod to create an additional Ray Serve replica to match the `num_replicas` configuration. Because the `max_replicas_per_node` is `1`, the new Ray Serve replica is running on the worker Pod without any replicas. After that, KubeRay marks the worker Pod as ready.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
After reconfiguration, KubeRay requests the head Pod to create an additional Ray Serve replica to match the `num_replicas` configuration. Because the `max_replicas_per_node` is `1`, the new Ray Serve replica runs on the worker Pod without any replicas. After that, KubeRay marks the worker Pod as ready.
```
created_at: 2025-03-20T16:51:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,195 @@
+(kuberay-rayservice-no-ray-serve-replica)=
+
+# Why are some RayService worker Pods not ready?
+
+This guide explores a specific scenario in KubeRay's RayService API where a Ray worker Pod remains in an unready state due to the absence of a Ray Serve replica.
+
+To better understand this section, you should be familiar with the following Ray Serve components: 
+the [Ray Serve replica and ProxyActor](https://docs.ray.io/en/latest/serve/architecture.html#high-level-view).
+
+ProxyActor is responsible for forwarding incoming requests to the corresponding Ray Serve replicas. 
+Hence, if a Ray Pod without a running ProxyActor receives requests, those requests will fail.
+KubeRay's readiness probe fails, rendering the Pods unready and preventing requests from being sent to them.
+
+The default behavior of Ray Serve only creates ProxyActor on Ray Pods with running Ray Serve replicas.
+See the following example for more details.
path: doc/source/cluster/kubernetes/user-guides/rayservice-no-ray-serve-replica.md
user: login: angelinalg
body: ```suggestion
To illustrate, the following example serves one simple Ray Serve app using RayService.
```
created_at: 2025-03-20T16:56:52Z
author_association: CONTRIBUTOR

PR comments:
user: login: kevin85421
created_at: 2025-03-08T01:11:38Z
body: cc @rueian 
author_association: MEMBER
user: login: kenchung285
created_at: 2025-03-10T23:51:31Z
body:  @MortalHappiness Could you review this PR?
author_association: CONTRIBUTOR
user: login: kevin85421
created_at: 2025-03-11T00:03:38Z
body: I will review the PR after @MortalHappiness @rueian review it. Btw, @kenchung285 have you installed Vale https://docs.ray.io/en/latest/ray-contribute/docs.html#how-to-use-vale? If not, would you mind installing it?
author_association: MEMBER
user: login: kenchung285
created_at: 2025-03-11T04:52:04Z
body: > I will review the PR after @MortalHappiness @rueian review it. Btw, @kenchung285 have you installed Vale https://docs.ray.io/en/latest/ray-contribute/docs.html#how-to-use-vale? If not, would you mind installing it?

I will install it later after work. Thanks for reminder
author_association: CONTRIBUTOR
user: login: kenchung285
created_at: 2025-03-11T14:35:00Z
body: @rueian The document has been updated, can you review it again?
author_association: CONTRIBUTOR
user: login: kenchung285
created_at: 2025-03-12T06:00:27Z
body: @kevin85421 I have update the document with vale tool check. Could you review the PR?
author_association: CONTRIBUTOR
user: login: kevin85421
created_at: 2025-03-18T03:59:07Z
body: I have already requested Ray doc team to review this PR.
author_association: MEMBER
user: login: kevin85421
created_at: 2025-03-22T04:50:50Z
body: cc @jjyao @edoakes would you mind merging this PR when you have time? Thanks!
author_association: MEMBER