Repository: kubernetes-sigs/gateway-api-inference-extension

Issue:
With flow control, EPP's CPU usage reaches 100%
When I was using the experimental flow control feature, based on my observation of the pod metrics, I noticed that the EPP's CPU usage would quickly reach nearly 100% after receiving the first request, approaching its limit. This situation will continue even if EPP doesn't receive any requests.

<img width="840" height="240" alt="Image" src="https://github.com/user-attachments/assets/c49d12b8-417f-4b02-a443-51c2b2d12b54" />

At the same time, I occasionally observe that the EPP restarts due to health check failures, not sure if it is affected by this phenomenon.

By inspecting the code, I found that it might be due to the ShardProcessor continuously invoking `runtime.Gosched()`. I can understand the design intent, but it seems that calling `runtime.Gosched()` too frequently could lead to high CPU usage.

https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/4752d66e372944df44d509d678a6b972fcb002f1/pkg/epp/flowcontrol/controller/internal/processor.go#L202

I attempted to make the following changes by introducing a ticker with a small time interval to avoid continuous calls. 

```go
func (sp *ShardProcessor) Run(ctx context.Context) {
	sp.logger.V(logutil.DEFAULT).Info("Shard processor run loop starting.")
	defer sp.logger.V(logutil.DEFAULT).Info("Shard processor run loop stopped.")

	sp.wg.Add(1)
	go sp.runCleanupSweep(ctx)

	// Create a ticker for periodic dispatch attempts to avoid tight loops
	dispatchTicker := sp.clock.NewTicker(time.Millisecond) // we may adjust interval as needed
	defer dispatchTicker.Stop()

	// ...
	for {
		select {
		case <-ctx.Done():
			sp.shutdown()
			sp.wg.Wait()
			return
		case item, ok := <-sp.enqueueChan:
			if !ok { // Should not happen in practice, but is a clean shutdown signal.
				sp.shutdown()
				sp.wg.Wait()
				return
			}
			// This is a safeguard against logic errors in the distributor.
			if item == nil {
				sp.logger.Error(nil, "Logic error: nil item received on shard processor enqueue channel, ignoring.")
				continue
			}
			sp.enqueue(item)
			sp.dispatchCycle(ctx) // Process immediately when an item arrives
		case <-dispatchTicker.C():
			sp.dispatchCycle(ctx) // Periodically attempt to dispatch from queues
		}
	}
}
``` 

This can reduce the CPU usage of EPP to about 2.5% when it is idle.

<img width="749" height="230" alt="Image" src="https://github.com/user-attachments/assets/b4539dc3-37f9-401c-8e04-38d05b6ea04d" />

But this introduces a behavior change: ShardProcessor will not continuously attempt to perform dispatch. However, I think it might be fine, because the saturation detector is unlikely to change its detection result within a very short period.

Some opinions from the maintainer are needed: 
* Should this phenomenon be considered a problem? (since I have not yet tested its impact on EPP performance)
* Does the aforementioned modification still align with the design intent of Flow control?

cc @LukeAVanDrie 

Discussion:
@LukeAVanDrie: Thanks for identifying this! Moving away from a busy loop seems best here. 

> But this introduces a behavior change: ShardProcessor will not continuously attempt to perform dispatch. However, I think it might be fine, because the saturation detector is unlikely to change its detection result within a very short period.

This should definitely be okay. There is some sustainable rate the inference pool can serve requests while staying below saturation. Your 1ms ticker means in the best case we can dispatch at 1000Hz. This is a reasonable upper bound for now, and the ticker is a pragmatic solution.

Long term, I plan on breaking the busy loop and making this more event-driven. But this is a well-reasoned, simple fix for now.@LukeAVanDrie: > Should this phenomenon be considered a problem? (since I have not yet tested its impact on EPP performance)

FWIW, in the load testing I have done (serving 400+ QPS) with the default scheduling profile this hasn't impacted EPP-induced request overhead latency, but it is resource wasteful.@LukeAVanDrie: Would you like to implement your fix? Your suggested code change LGTM. I am also happy to do so on your behalf.@delavet: > Would you like to implement your fix? Your suggested code change LGTM. I am also happy to do so on your behalf.

Of course! Sorry for the delay, there might be some differences in working hours. :)

/assign

Solution patch:
diff --git a/apix/config/v1alpha1/endpointpickerconfig_types.go b/apix/config/v1alpha1/endpointpickerconfig_types.go
index 705e372..8e80e9d 100644
--- a/apix/config/v1alpha1/endpointpickerconfig_types.go
+++ b/apix/config/v1alpha1/endpointpickerconfig_types.go
@@ -39,13 +39,25 @@ type EndpointPickerConfig struct {
 	// SchedulingProfiles is the list of named SchedulingProfiles
 	// that will be created.
 	SchedulingProfiles []SchedulingProfile `json:"schedulingProfiles"`
+
+	// +optional
+	// FeatureGates is a set of flags that enable various experimental features with the EPP.
+	// If omitted non of these experimental features will be enabled.
+	FeatureGates FeatureGates `json:"featureGates,omitempty"`
+
+	// +optional
+	// SaturationDetector when present specifies the configuration of the
+	// Saturation detector. If not present, default values are used.
+	SaturationDetector *SaturationDetector `json:"saturationDetector,omitempty"`
 }
 
 func (cfg EndpointPickerConfig) String() string {
 	return fmt.Sprintf(
-		"{Plugins: %v, SchedulingProfiles: %v}",
+		"{Plugins: %v, SchedulingProfiles: %v, FeatureGates: %v, SaturationDetector: %v}",
 		cfg.Plugins,
 		cfg.SchedulingProfiles,
+		cfg.FeatureGates,
+		cfg.SaturationDetector,
 	)
 }
 
@@ -118,3 +130,64 @@ func (sp SchedulingPlugin) String() string {
 	}
 	return fmt.Sprintf("{PluginRef: %s%s}", sp.PluginRef, weight)
 }
+
+// FeatureGates is a set of flags that enable various experimental features with the EPP
+type FeatureGates []string
+
+func (fg FeatureGates) String() string {
+	if fg == nil {
+		return "{}"
+	}
+
+	result := ""
+	for _, gate := range fg {
+		result += gate + ","
+	}
+
+	if len(result) > 0 {
+		result = result[:len(result)-1]
+	}
+	return "{" + result + "}"
+}
+
+// SaturationDetector
+type SaturationDetector struct {
+	// +optional
+	// QueueDepthThreshold defines the backend waiting queue size above which a
+	// pod is considered to have insufficient capacity for new requests.
+	QueueDepthThreshold int `json:"queueDepthThreshold,omitempty"`
+
+	// +optional
+	// KVCacheUtilThreshold defines the KV cache utilization (0.0 to 1.0) above
+	// which a pod is considered to have insufficient capacity.
+	KVCacheUtilThreshold float64 `json:"kvCacheUtilThreshold,omitempty"`
+
+	// +optional
+	// MetricsStalenessThreshold defines how old a pod's metrics can be.
+	// If a pod's metrics are older than this, it might be excluded from
+	// "good capacity" considerations or treated as having no capacity for
+	// safety.
+	MetricsStalenessThreshold metav1.Duration `json:"metricsStalenessThreshold,omitempty"`
+}
+
+func (sd *SaturationDetector) String() string {
+	result := ""
+	if sd != nil {
+		if sd.QueueDepthThreshold != 0 {
+			result += fmt.Sprintf("QueueDepthThreshold: %d", sd.QueueDepthThreshold)
+		}
+		if sd.KVCacheUtilThreshold != 0.0 {
+			if len(result) != 0 {
+				result += ", "
+			}
+			result += fmt.Sprintf("KVCacheUtilThreshold: %g", sd.KVCacheUtilThreshold)
+		}
+		if sd.MetricsStalenessThreshold.Duration != 0.0 {
+			if len(result) != 0 {
+				result += ", "
+			}
+			result += fmt.Sprintf("MetricsStalenessThreshold: %s", sd.MetricsStalenessThreshold)
+		}
+	}
+	return "{" + result + "}"
+}
diff --git a/apix/config/v1alpha1/zz_generated.deepcopy.go b/apix/config/v1alpha1/zz_generated.deepcopy.go
index 1326f35..701f737 100644
--- a/apix/config/v1alpha1/zz_generated.deepcopy.go
+++ b/apix/config/v1alpha1/zz_generated.deepcopy.go
@@ -43,6 +43,16 @@ func (in *EndpointPickerConfig) DeepCopyInto(out *EndpointPickerConfig) {
 			(*in)[i].DeepCopyInto(&(*out)[i])
 		}
 	}
+	if in.FeatureGates != nil {
+		in, out := &in.FeatureGates, &out.FeatureGates
+		*out = make(FeatureGates, len(*in))
+		copy(*out, *in)
+	}
+	if in.SaturationDetector != nil {
+		in, out := &in.SaturationDetector, &out.SaturationDetector
+		*out = new(SaturationDetector)
+		**out = **in
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EndpointPickerConfig.
@@ -63,6 +73,25 @@ func (in *EndpointPickerConfig) DeepCopyObject() runtime.Object {
 	return nil
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in FeatureGates) DeepCopyInto(out *FeatureGates) {
+	{
+		in := &in
+		*out = make(FeatureGates, len(*in))
+		copy(*out, *in)
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new FeatureGates.
+func (in FeatureGates) DeepCopy() FeatureGates {
+	if in == nil {
+		return nil
+	}
+	out := new(FeatureGates)
+	in.DeepCopyInto(out)
+	return *out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *PluginSpec) DeepCopyInto(out *PluginSpec) {
 	*out = *in
@@ -83,6 +112,22 @@ func (in *PluginSpec) DeepCopy() *PluginSpec {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *SaturationDetector) DeepCopyInto(out *SaturationDetector) {
+	*out = *in
+	out.MetricsStalenessThreshold = in.MetricsStalenessThreshold
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaturationDetector.
+func (in *SaturationDetector) DeepCopy() *SaturationDetector {
+	if in == nil {
+		return nil
+	}
+	out := new(SaturationDetector)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *SchedulingPlugin) DeepCopyInto(out *SchedulingPlugin) {
 	*out = *in
diff --git a/apix/v1alpha2/inferencemodelrewrite_types.go b/apix/v1alpha2/inferencemodelrewrite_types.go
new file mode 100644
index 0000000..ef68d83
--- /dev/null
+++ b/apix/v1alpha2/inferencemodelrewrite_types.go
@@ -0,0 +1,194 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package v1alpha2
+
+import (
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+// InferenceModelRewrite is the Schema for the InferenceModelRewrite API.
+// +kubebuilder:object:root=true
+// +kubebuilder:subresource:status
+// +kubebuilder:printcolumn:name="Inference Pool",type=string,JSONPath=`.spec.poolRef.name`
+// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
+// +genclient
+type InferenceModelRewrite struct {
+	metav1.TypeMeta   `json:",inline"`
+	metav1.ObjectMeta `json:"metadata,omitempty"`
+
+	Spec   InferenceModelRewriteSpec   `json:"spec,omitempty"`
+	Status InferenceModelRewriteStatus `json:"status,omitempty"`
+}
+
+// InferenceModelRewriteList contains a list of InferenceModelRewrite.
+//
+// +kubebuilder:object:root=true
+type InferenceModelRewriteList struct {
+	metav1.TypeMeta `json:",inline"`
+	metav1.ListMeta `json:"metadata,omitempty"`
+	Items           []InferenceModelRewrite `json:"items"`
+}
+
+// InferenceModelRewriteSpec defines the desired state of InferenceModelRewrite.
+type InferenceModelRewriteSpec struct {
+	// PoolRef is a reference to the inference pool.
+	// +kubebuilder:validation:Required
+	PoolRef *PoolObjectReference `json:"poolRef"`
+
+	// Rules are the ordered set of rules for rewriting inference requests.
+	// The first rule to match a request will be used.
+
+	//
+	// --- Precedence and Conflict Resolution ---
+	// If multiple InferenceModelRewrite resources target the same
+	// InferencePool, the controller will merge them based on precedence.
+	//
+	// **Timestamp Wins:** If two rules from different rewrites all matches,
+	// the rule from the *oldest*
+	// InferenceModelRewrite resource (determined by
+	// metadata.creationTimestamp) will be used.
+	// +required
+	Rules []InferenceModelRewriteRule `json:"rules"`
+}
+
+// InferenceModelRewriteRule defines the match criteria and corresponding action.
+//
+// A specific model name can only be matched by one rule across all
+// rules attached to the same InferencePool. If multiple rules attempt
+// to match the same model name, the oldest rule (by creationTimestamp)
+// will be the only one considered valid.
+type InferenceModelRewriteRule struct {
+	// Matches defines the criteria for matching a request.
+	// If multiple match criteria are specified, a request matches if
+	// ANY of the criteria are satisfied (logical OR).
+	// If empty, the rule matches all requests.
+
+	// +optional
+	Matches []Match `json:"matches,omitempty"`
+
+	// --- Actions ---
+	// Targets defines how to distribute traffic across a set of
+	// weighted model targets. This is used for traffic splitting, A/B tests,
+	// or canary rollouts.
+	// +optional
+	// +kubebuilder:validation:MinItems=1
+	//
+	Targets []TargetModel `json:"split,omitempty"`
+}
+
+// TargetModel defines a weighted model destination for traffic distribution.
+type TargetModel struct {
+	// (The following comment is copied from the original targetModel)
+	// Weight is used to determine the proportion of traffic that should be
+	// sent to this model when multiple target models are specified.
+	//
+	// Weight defines the proportion of requests forwarded to the specified
+	// model. This is computed as weight/(sum of all weights in this
+	// TargetModels list). For non-zero values, there may be some epsilon from
+	// the exact proportion defined here depending on the precision an
+	// implementation supports. Weight is not a percentage and the sum of
+	// weights does not need to equal 100.
+	//
+	// If a weight is set for any targetModel, it must be set for all targetModels.
+	// Conversely weights are optional, so long as ALL targetModels do not specify a weight.
+	//
+	// +optional
+	// +kubebuilder:validation:Minimum=1
+	// +kubebuilder:validation:Maximum=1000000
+	Weight int32 `json:"weight"`
+
+	// --- Destination Types ---
+	// ModelRewrite specifies a static model name destination.
+	// +required
+	ModelRewrite string `json:"modelRewrite"`
+}
+
+// Match defines the criteria for matching the LLM requests.
+type Match struct {
+	// Model specifies the criteria for matching the 'model' field
+	// within the JSON request body.
+	// +required
+	Model *ModelMatch `json:"model,omitempty"`
+}
+
+// ModelMatch defines how to match against the model name in the request body.
+type ModelMatch struct {
+	// Type specifies the kind of string matching to use.
+	// Supported value is "Exact". Defaults to "Exact".
+	// +optional
+	// +kubebuilder:default=Exact
+	Type *MatchValidationType `json:"type,omitempty"`
+
+	// Value is the model name string to match against.
+	// +required
+	// +kubebuilder:validation:MinLength=1
+	Value string `json:"value"`
+}
+
+// MatchValidationType specifies the type of string matching to use.
+// +kubebuilder:validation:Enum=Exact
+type MatchValidationType string
+
+const (
+	// MatchExact indicates that the model name must match exactly.
+	MatchExact MatchValidationType = "Exact"
+)
+
+// InferenceModelRewriteStatus defines the observed state of InferenceModelRewrite.
+type InferenceModelRewriteStatus struct {
+	// Conditions track the state of the InferenceModelRewrite.
+	//
+	// Known condition types are:
+	//
+	// * "Accepted"
+	//
+	// +optional
+	// +listType=map
+	// +listMapKey=type
+	// +kubebuilder:validation:MaxItems=8
+	// +kubebuilder:default={{type: "Accepted", status: "Unknown", reason:"Pending", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}
+	Conditions []metav1.Condition `json:"conditions,omitempty"`
+}
+
+// InferenceModelRewriteConditionType is a type of condition for the InferenceModelRewrite.
+type InferenceModelRewriteConditionType string
+
+// InferenceModelRewriteConditionReason is the reason for a given InferenceModelRewriteConditionType.
+type InferenceModelRewriteConditionReason string
+
+const (
+	// RewriteConditionAccepted indicates if the rewrite is accepted, and if not, why.
+	// This is the primary condition for this resource.
+	//
+	// Possible reasons for this condition to be True are:
+	//
+	// * "Accepted"
+	//
+	// Possible reasons for this condition to be Unknown are:
+	//
+	// * "Pending"
+	//
+	RewriteConditionAccepted InferenceModelRewriteConditionType = "Accepted"
+
+	// RewriteReasonAccepted indicates the rewrite is valid, non-conflicting,
+	// and has been successfully applied to the inference pool.
+	RewriteReasonAccepted InferenceModelRewriteConditionReason = "Accepted"
+
+	// RewriteReasonPending is the initial state, and indicates that the
+	// controller has not yet reconciled the InferenceModelRewrite.
+	RewriteReasonPending InferenceModelRewriteConditionReason = "Pending"
+)
diff --git a/apix/v1alpha2/inferenceobjective_types.go b/apix/v1alpha2/inferenceobjective_types.go
index 691b881..ecd50b3 100644
--- a/apix/v1alpha2/inferenceobjective_types.go
+++ b/apix/v1alpha2/inferenceobjective_types.go
@@ -78,27 +78,6 @@ type InferenceObjectiveSpec struct {
 	PoolRef PoolObjectReference `json:"poolRef"`
 }
 
-// PoolObjectReference identifies an API object within the namespace of the
-// referrer.
-type PoolObjectReference struct {
-	// Group is the group of the referent.
-	//
-	// +optional
-	// +kubebuilder:default="inference.networking.k8s.io"
-	Group Group `json:"group,omitempty"`
-
-	// Kind is kind of the referent. For example "InferencePool".
-	//
-	// +optional
-	// +kubebuilder:default="InferencePool"
-	Kind Kind `json:"kind,omitempty"`
-
-	// Name is the name of the referent.
-	//
-	// +kubebuilder:validation:Required
-	Name ObjectName `json:"name"`
-}
-
 // InferenceObjectiveStatus defines the observed state of InferenceObjective
 type InferenceObjectiveStatus struct {
 	// Conditions track the state of the InferenceObjective.
diff --git a/apix/v1alpha2/shared_types.go b/apix/v1alpha2/shared_types.go
index 1fe46dd..4fc2c77 100644
--- a/apix/v1alpha2/shared_types.go
+++ b/apix/v1alpha2/shared_types.go
@@ -127,3 +127,24 @@ type LabelKey string
 // +kubebuilder:validation:MaxLength=63
 // +kubebuilder:validation:Pattern=`^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?$`
 type LabelValue string
+
+// PoolObjectReference identifies an API object within the namespace of the
+// referrer.
+type PoolObjectReference struct {
+	// Group is the group of the referent.
+	//
+	// +optional
+	// +kubebuilder:default="inference.networking.k8s.io"
+	Group Group `json:"group,omitempty"`
+
+	// Kind is kind of the referent. For example "InferencePool".
+	//
+	// +optional
+	// +kubebuilder:default="InferencePool"
+	Kind Kind `json:"kind,omitempty"`
+
+	// Name is the name of the referent.
+	//
+	// +kubebuilder:validation:Required
+	Name ObjectName `json:"name"`
+}
diff --git a/apix/v1alpha2/zz_generated.deepcopy.go b/apix/v1alpha2/zz_generated.deepcopy.go
index 0249d44..237194d 100644
--- a/apix/v1alpha2/zz_generated.deepcopy.go
+++ b/apix/v1alpha2/zz_generated.deepcopy.go
@@ -60,6 +60,141 @@ func (in *Extension) DeepCopy() *Extension {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *InferenceModelRewrite) DeepCopyInto(out *InferenceModelRewrite) {
+	*out = *in
+	out.TypeMeta = in.TypeMeta
+	in.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
+	in.Spec.DeepCopyInto(&out.Spec)
+	in.Status.DeepCopyInto(&out.Status)
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceModelRewrite.
+func (in *InferenceModelRewrite) DeepCopy() *InferenceModelRewrite {
+	if in == nil {
+		return nil
+	}
+	out := new(InferenceModelRewrite)
+	in.DeepCopyInto(out)
+	return out
+}
+
+// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
+func (in *InferenceModelRewrite) DeepCopyObject() runtime.Object {
+	if c := in.DeepCopy(); c != nil {
+		return c
+	}
+	return nil
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *InferenceModelRewriteList) DeepCopyInto(out *InferenceModelRewriteList) {
+	*out = *in
+	out.TypeMeta = in.TypeMeta
+	in.ListMeta.DeepCopyInto(&out.ListMeta)
+	if in.Items != nil {
+		in, out := &in.Items, &out.Items
+		*out = make([]InferenceModelRewrite, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceModelRewriteList.
+func (in *InferenceModelRewriteList) DeepCopy() *InferenceModelRewriteList {
+	if in == nil {
+		return nil
+	}
+	out := new(InferenceModelRewriteList)
+	in.DeepCopyInto(out)
+	return out
+}
+
+// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
+func (in *InferenceModelRewriteList) DeepCopyObject() runtime.Object {
+	if c := in.DeepCopy(); c != nil {
+		return c
+	}
+	return nil
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *InferenceModelRewriteRule) DeepCopyInto(out *InferenceModelRewriteRule) {
+	*out = *in
+	if in.Matches != nil {
+		in, out := &in.Matches, &out.Matches
+		*out = make([]Match, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
+	if in.Targets != nil {
+		in, out := &in.Targets, &out.Targets
+		*out = make([]TargetModel, len(*in))
+		copy(*out, *in)
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceModelRewriteRule.
+func (in *InferenceModelRewriteRule) DeepCopy() *InferenceModelRewriteRule {
+	if in == nil {
+		return nil
+	}
+	out := new(InferenceModelRewriteRule)
+	in.DeepCopyInto(out)
+	return out
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *InferenceModelRewriteSpec) DeepCopyInto(out *InferenceModelRewriteSpec) {
+	*out = *in
+	if in.PoolRef != nil {
+		in, out := &in.PoolRef, &out.PoolRef
+		*out = new(PoolObjectReference)
+		**out = **in
+	}
+	if in.Rules != nil {
+		in, out := &in.Rules, &out.Rules
+		*out = make([]InferenceModelRewriteRule, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceModelRewriteSpec.
+func (in *InferenceModelRewriteSpec) DeepCopy() *InferenceModelRewriteSpec {
+	if in == nil {
+		return nil
+	}
+	out := new(InferenceModelRewriteSpec)
+	in.DeepCopyInto(out)
+	return out
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *InferenceModelRewriteStatus) DeepCopyInto(out *InferenceModelRewriteStatus) {
+	*out = *in
+	if in.Conditions != nil {
+		in, out := &in.Conditions, &out.Conditions
+		*out = make([]v1.Condition, len(*in))
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceModelRewriteStatus.
+func (in *InferenceModelRewriteStatus) DeepCopy() *InferenceModelRewriteStatus {
+	if in == nil {
+		return nil
+	}
+	out := new(InferenceModelRewriteStatus)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *InferenceObjective) DeepCopyInto(out *InferenceObjective) {
 	*out = *in
@@ -266,6 +401,46 @@ func (in *InferencePoolStatus) DeepCopy() *InferencePoolStatus {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *Match) DeepCopyInto(out *Match) {
+	*out = *in
+	if in.Model != nil {
+		in, out := &in.Model, &out.Model
+		*out = new(ModelMatch)
+		(*in).DeepCopyInto(*out)
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Match.
+func (in *Match) DeepCopy() *Match {
+	if in == nil {
+		return nil
+	}
+	out := new(Match)
+	in.DeepCopyInto(out)
+	return out
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *ModelMatch) DeepCopyInto(out *ModelMatch) {
+	*out = *in
+	if in.Type != nil {
+		in, out := &in.Type, &out.Type
+		*out = new(MatchValidationType)
+		**out = **in
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ModelMatch.
+func (in *ModelMatch) DeepCopy() *ModelMatch {
+	if in == nil {
+		return nil
+	}
+	out := new(ModelMatch)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *ParentGatewayReference) DeepCopyInto(out *ParentGatewayReference) {
 	*out = *in
@@ -333,3 +508,18 @@ func (in *PoolStatus) DeepCopy() *PoolStatus {
 	in.DeepCopyInto(out)
 	return out
 }
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *TargetModel) DeepCopyInto(out *TargetModel) {
+	*out = *in
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TargetModel.
+func (in *TargetModel) DeepCopy() *TargetModel {
+	if in == nil {
+		return nil
+	}
+	out := new(TargetModel)
+	in.DeepCopyInto(out)
+	return out
+}
diff --git a/apix/v1alpha2/zz_generated.register.go b/apix/v1alpha2/zz_generated.register.go
index ae0e1da..42e5695 100644
--- a/apix/v1alpha2/zz_generated.register.go
+++ b/apix/v1alpha2/zz_generated.register.go
@@ -61,6 +61,8 @@ func init() {
 // Adds the list of known types to Scheme.
 func addKnownTypes(scheme *runtime.Scheme) error {
 	scheme.AddKnownTypes(SchemeGroupVersion,
+		&InferenceModelRewrite{},
+		&InferenceModelRewriteList{},
 		&InferenceObjective{},
 		&InferenceObjectiveList{},
 		&InferencePool{},
diff --git a/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewrite.go b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewrite.go
new file mode 100644
index 0000000..506e795
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewrite.go
@@ -0,0 +1,242 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	types "k8s.io/apimachinery/pkg/types"
+	v1 "k8s.io/client-go/applyconfigurations/meta/v1"
+)
+
+// InferenceModelRewriteApplyConfiguration represents a declarative configuration of the InferenceModelRewrite type for use
+// with apply.
+type InferenceModelRewriteApplyConfiguration struct {
+	v1.TypeMetaApplyConfiguration    `json:",inline"`
+	*v1.ObjectMetaApplyConfiguration `json:"metadata,omitempty"`
+	Spec                             *InferenceModelRewriteSpecApplyConfiguration   `json:"spec,omitempty"`
+	Status                           *InferenceModelRewriteStatusApplyConfiguration `json:"status,omitempty"`
+}
+
+// InferenceModelRewrite constructs a declarative configuration of the InferenceModelRewrite type for use with
+// apply.
+func InferenceModelRewrite(name, namespace string) *InferenceModelRewriteApplyConfiguration {
+	b := &InferenceModelRewriteApplyConfiguration{}
+	b.WithName(name)
+	b.WithNamespace(namespace)
+	b.WithKind("InferenceModelRewrite")
+	b.WithAPIVersion("inference.networking.x-k8s.io/v1alpha2")
+	return b
+}
+func (b InferenceModelRewriteApplyConfiguration) IsApplyConfiguration() {}
+
+// WithKind sets the Kind field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Kind field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithKind(value string) *InferenceModelRewriteApplyConfiguration {
+	b.TypeMetaApplyConfiguration.Kind = &value
+	return b
+}
+
+// WithAPIVersion sets the APIVersion field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the APIVersion field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithAPIVersion(value string) *InferenceModelRewriteApplyConfiguration {
+	b.TypeMetaApplyConfiguration.APIVersion = &value
+	return b
+}
+
+// WithName sets the Name field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Name field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithName(value string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.Name = &value
+	return b
+}
+
+// WithGenerateName sets the GenerateName field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the GenerateName field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithGenerateName(value string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.GenerateName = &value
+	return b
+}
+
+// WithNamespace sets the Namespace field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Namespace field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithNamespace(value string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.Namespace = &value
+	return b
+}
+
+// WithUID sets the UID field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the UID field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithUID(value types.UID) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.UID = &value
+	return b
+}
+
+// WithResourceVersion sets the ResourceVersion field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the ResourceVersion field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithResourceVersion(value string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.ResourceVersion = &value
+	return b
+}
+
+// WithGeneration sets the Generation field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Generation field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithGeneration(value int64) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.Generation = &value
+	return b
+}
+
+// WithCreationTimestamp sets the CreationTimestamp field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the CreationTimestamp field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithCreationTimestamp(value metav1.Time) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.CreationTimestamp = &value
+	return b
+}
+
+// WithDeletionTimestamp sets the DeletionTimestamp field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the DeletionTimestamp field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithDeletionTimestamp(value metav1.Time) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.DeletionTimestamp = &value
+	return b
+}
+
+// WithDeletionGracePeriodSeconds sets the DeletionGracePeriodSeconds field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the DeletionGracePeriodSeconds field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithDeletionGracePeriodSeconds(value int64) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	b.ObjectMetaApplyConfiguration.DeletionGracePeriodSeconds = &value
+	return b
+}
+
+// WithLabels puts the entries into the Labels field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, the entries provided by each call will be put on the Labels field,
+// overwriting an existing map entries in Labels field with the same key.
+func (b *InferenceModelRewriteApplyConfiguration) WithLabels(entries map[string]string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	if b.ObjectMetaApplyConfiguration.Labels == nil && len(entries) > 0 {
+		b.ObjectMetaApplyConfiguration.Labels = make(map[string]string, len(entries))
+	}
+	for k, v := range entries {
+		b.ObjectMetaApplyConfiguration.Labels[k] = v
+	}
+	return b
+}
+
+// WithAnnotations puts the entries into the Annotations field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, the entries provided by each call will be put on the Annotations field,
+// overwriting an existing map entries in Annotations field with the same key.
+func (b *InferenceModelRewriteApplyConfiguration) WithAnnotations(entries map[string]string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	if b.ObjectMetaApplyConfiguration.Annotations == nil && len(entries) > 0 {
+		b.ObjectMetaApplyConfiguration.Annotations = make(map[string]string, len(entries))
+	}
+	for k, v := range entries {
+		b.ObjectMetaApplyConfiguration.Annotations[k] = v
+	}
+	return b
+}
+
+// WithOwnerReferences adds the given value to the OwnerReferences field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, values provided by each call will be appended to the OwnerReferences field.
+func (b *InferenceModelRewriteApplyConfiguration) WithOwnerReferences(values ...*v1.OwnerReferenceApplyConfiguration) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	for i := range values {
+		if values[i] == nil {
+			panic("nil value passed to WithOwnerReferences")
+		}
+		b.ObjectMetaApplyConfiguration.OwnerReferences = append(b.ObjectMetaApplyConfiguration.OwnerReferences, *values[i])
+	}
+	return b
+}
+
+// WithFinalizers adds the given value to the Finalizers field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, values provided by each call will be appended to the Finalizers field.
+func (b *InferenceModelRewriteApplyConfiguration) WithFinalizers(values ...string) *InferenceModelRewriteApplyConfiguration {
+	b.ensureObjectMetaApplyConfigurationExists()
+	for i := range values {
+		b.ObjectMetaApplyConfiguration.Finalizers = append(b.ObjectMetaApplyConfiguration.Finalizers, values[i])
+	}
+	return b
+}
+
+func (b *InferenceModelRewriteApplyConfiguration) ensureObjectMetaApplyConfigurationExists() {
+	if b.ObjectMetaApplyConfiguration == nil {
+		b.ObjectMetaApplyConfiguration = &v1.ObjectMetaApplyConfiguration{}
+	}
+}
+
+// WithSpec sets the Spec field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Spec field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithSpec(value *InferenceModelRewriteSpecApplyConfiguration) *InferenceModelRewriteApplyConfiguration {
+	b.Spec = value
+	return b
+}
+
+// WithStatus sets the Status field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Status field is set to the value of the last call.
+func (b *InferenceModelRewriteApplyConfiguration) WithStatus(value *InferenceModelRewriteStatusApplyConfiguration) *InferenceModelRewriteApplyConfiguration {
+	b.Status = value
+	return b
+}
+
+// GetKind retrieves the value of the Kind field in the declarative configuration.
+func (b *InferenceModelRewriteApplyConfiguration) GetKind() *string {
+	return b.TypeMetaApplyConfiguration.Kind
+}
+
+// GetAPIVersion retrieves the value of the APIVersion field in the declarative configuration.
+func (b *InferenceModelRewriteApplyConfiguration) GetAPIVersion() *string {
+	return b.TypeMetaApplyConfiguration.APIVersion
+}
+
+// GetName retrieves the value of the Name field in the declarative configuration.
+func (b *InferenceModelRewriteApplyConfiguration) GetName() *string {
+	b.ensureObjectMetaApplyConfigurationExists()
+	return b.ObjectMetaApplyConfiguration.Name
+}
+
+// GetNamespace retrieves the value of the Namespace field in the declarative configuration.
+func (b *InferenceModelRewriteApplyConfiguration) GetNamespace() *string {
+	b.ensureObjectMetaApplyConfigurationExists()
+	return b.ObjectMetaApplyConfiguration.Namespace
+}
diff --git a/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewriterule.go b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewriterule.go
new file mode 100644
index 0000000..7e03c19
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewriterule.go
@@ -0,0 +1,58 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+// InferenceModelRewriteRuleApplyConfiguration represents a declarative configuration of the InferenceModelRewriteRule type for use
+// with apply.
+type InferenceModelRewriteRuleApplyConfiguration struct {
+	Matches []MatchApplyConfiguration       `json:"matches,omitempty"`
+	Targets []TargetModelApplyConfiguration `json:"split,omitempty"`
+}
+
+// InferenceModelRewriteRuleApplyConfiguration constructs a declarative configuration of the InferenceModelRewriteRule type for use with
+// apply.
+func InferenceModelRewriteRule() *InferenceModelRewriteRuleApplyConfiguration {
+	return &InferenceModelRewriteRuleApplyConfiguration{}
+}
+
+// WithMatches adds the given value to the Matches field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, values provided by each call will be appended to the Matches field.
+func (b *InferenceModelRewriteRuleApplyConfiguration) WithMatches(values ...*MatchApplyConfiguration) *InferenceModelRewriteRuleApplyConfiguration {
+	for i := range values {
+		if values[i] == nil {
+			panic("nil value passed to WithMatches")
+		}
+		b.Matches = append(b.Matches, *values[i])
+	}
+	return b
+}
+
+// WithTargets adds the given value to the Targets field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, values provided by each call will be appended to the Targets field.
+func (b *InferenceModelRewriteRuleApplyConfiguration) WithTargets(values ...*TargetModelApplyConfiguration) *InferenceModelRewriteRuleApplyConfiguration {
+	for i := range values {
+		if values[i] == nil {
+			panic("nil value passed to WithTargets")
+		}
+		b.Targets = append(b.Targets, *values[i])
+	}
+	return b
+}
diff --git a/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewritespec.go b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewritespec.go
new file mode 100644
index 0000000..0182234
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewritespec.go
@@ -0,0 +1,53 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+// InferenceModelRewriteSpecApplyConfiguration represents a declarative configuration of the InferenceModelRewriteSpec type for use
+// with apply.
+type InferenceModelRewriteSpecApplyConfiguration struct {
+	PoolRef *PoolObjectReferenceApplyConfiguration        `json:"poolRef,omitempty"`
+	Rules   []InferenceModelRewriteRuleApplyConfiguration `json:"rules,omitempty"`
+}
+
+// InferenceModelRewriteSpecApplyConfiguration constructs a declarative configuration of the InferenceModelRewriteSpec type for use with
+// apply.
+func InferenceModelRewriteSpec() *InferenceModelRewriteSpecApplyConfiguration {
+	return &InferenceModelRewriteSpecApplyConfiguration{}
+}
+
+// WithPoolRef sets the PoolRef field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the PoolRef field is set to the value of the last call.
+func (b *InferenceModelRewriteSpecApplyConfiguration) WithPoolRef(value *PoolObjectReferenceApplyConfiguration) *InferenceModelRewriteSpecApplyConfiguration {
+	b.PoolRef = value
+	return b
+}
+
+// WithRules adds the given value to the Rules field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, values provided by each call will be appended to the Rules field.
+func (b *InferenceModelRewriteSpecApplyConfiguration) WithRules(values ...*InferenceModelRewriteRuleApplyConfiguration) *InferenceModelRewriteSpecApplyConfiguration {
+	for i := range values {
+		if values[i] == nil {
+			panic("nil value passed to WithRules")
+		}
+		b.Rules = append(b.Rules, *values[i])
+	}
+	return b
+}
diff --git a/client-go/applyconfiguration/apix/v1alpha2/match.go b/client-go/applyconfiguration/apix/v1alpha2/match.go
new file mode 100644
index 0000000..c0eca17
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/match.go
@@ -0,0 +1,39 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+// MatchApplyConfiguration represents a declarative configuration of the Match type for use
+// with apply.
+type MatchApplyConfiguration struct {
+	Model *ModelMatchApplyConfiguration `json:"model,omitempty"`
+}
+
+// MatchApplyConfiguration constructs a declarative configuration of the Match type for use with
+// apply.
+func Match() *MatchApplyConfiguration {
+	return &MatchApplyConfiguration{}
+}
+
+// WithModel sets the Model field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Model field is set to the value of the last call.
+func (b *MatchApplyConfiguration) WithModel(value *ModelMatchApplyConfiguration) *MatchApplyConfiguration {
+	b.Model = value
+	return b
+}
diff --git a/client-go/applyconfiguration/apix/v1alpha2/modelmatch.go b/client-go/applyconfiguration/apix/v1alpha2/modelmatch.go
new file mode 100644
index 0000000..2f26d41
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/modelmatch.go
@@ -0,0 +1,52 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	apixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
+)
+
+// ModelMatchApplyConfiguration represents a declarative configuration of the ModelMatch type for use
+// with apply.
+type ModelMatchApplyConfiguration struct {
+	Type  *apixv1alpha2.MatchValidationType `json:"type,omitempty"`
+	Value *string                           `json:"value,omitempty"`
+}
+
+// ModelMatchApplyConfiguration constructs a declarative configuration of the ModelMatch type for use with
+// apply.
+func ModelMatch() *ModelMatchApplyConfiguration {
+	return &ModelMatchApplyConfiguration{}
+}
+
+// WithType sets the Type field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Type field is set to the value of the last call.
+func (b *ModelMatchApplyConfiguration) WithType(value apixv1alpha2.MatchValidationType) *ModelMatchApplyConfiguration {
+	b.Type = &value
+	return b
+}
+
+// WithValue sets the Value field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Value field is set to the value of the last call.
+func (b *ModelMatchApplyConfiguration) WithValue(value string) *ModelMatchApplyConfiguration {
+	b.Value = &value
+	return b
+}
diff --git a/client-go/applyconfiguration/apix/v1alpha2/targetmodel.go b/client-go/applyconfiguration/apix/v1alpha2/targetmodel.go
new file mode 100644
index 0000000..4125a32
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/targetmodel.go
@@ -0,0 +1,48 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+// TargetModelApplyConfiguration represents a declarative configuration of the TargetModel type for use
+// with apply.
+type TargetModelApplyConfiguration struct {
+	Weight       *int32  `json:"weight,omitempty"`
+	ModelRewrite *string `json:"modelRewrite,omitempty"`
+}
+
+// TargetModelApplyConfiguration constructs a declarative configuration of the TargetModel type for use with
+// apply.
+func TargetModel() *TargetModelApplyConfiguration {
+	return &TargetModelApplyConfiguration{}
+}
+
+// WithWeight sets the Weight field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Weight field is set to the value of the last call.
+func (b *TargetModelApplyConfiguration) WithWeight(value int32) *TargetModelApplyConfiguration {
+	b.Weight = &value
+	return b
+}
+
+// WithModelRewrite sets the ModelRewrite field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the ModelRewrite field is set to the value of the last call.
+func (b *TargetModelApplyConfiguration) WithModelRewrite(value string) *TargetModelApplyConfiguration {
+	b.ModelRewrite = &value
+	return b
+}
diff --git a/client-go/applyconfiguration/utils.go b/client-go/applyconfiguration/utils.go
index 7e4ea09..93d50e5 100644
--- a/client-go/applyconfiguration/utils.go
+++ b/client-go/applyconfiguration/utils.go
@@ -66,6 +66,14 @@ func ForKind(kind schema.GroupVersionKind) interface{} {
 		// Group=inference.networking.x-k8s.io, Version=v1alpha2
 	case v1alpha2.SchemeGroupVersion.WithKind("Extension"):
 		return &apixv1alpha2.ExtensionApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("InferenceModelRewrite"):
+		return &apixv1alpha2.InferenceModelRewriteApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("InferenceModelRewriteRule"):
+		return &apixv1alpha2.InferenceModelRewriteRuleApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("InferenceModelRewriteSpec"):
+		return &apixv1alpha2.InferenceModelRewriteSpecApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("InferenceModelRewriteStatus"):
+		return &apixv1alpha2.InferenceModelRewriteStatusApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("InferenceObjective"):
 		return &apixv1alpha2.InferenceObjectiveApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("InferenceObjectiveSpec"):
@@ -78,12 +86,18 @@ func ForKind(kind schema.GroupVersionKind) interface{} {
 		return &apixv1alpha2.InferencePoolSpecApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("InferencePoolStatus"):
 		return &apixv1alpha2.InferencePoolStatusApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("Match"):
+		return &apixv1alpha2.MatchApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("ModelMatch"):
+		return &apixv1alpha2.ModelMatchApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("ParentGatewayReference"):
 		return &apixv1alpha2.ParentGatewayReferenceApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("PoolObjectReference"):
 		return &apixv1alpha2.PoolObjectReferenceApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("PoolStatus"):
 		return &apixv1alpha2.PoolStatusApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("TargetModel"):
+		return &apixv1alpha2.TargetModelApplyConfiguration{}
 
 	}
 	return nil
diff --git a/client-go/clientset/versioned/typed/apix/v1alpha2/apix_client.go b/client-go/clientset/versioned/typed/apix/v1alpha2/apix_client.go
index 84f636f..8d21b3c8 100644
--- a/client-go/clientset/versioned/typed/apix/v1alpha2/apix_client.go
+++ b/client-go/clientset/versioned/typed/apix/v1alpha2/apix_client.go
@@ -28,6 +28,7 @@ import (
 
 type XInferenceV1alpha2Interface interface {
 	RESTClient() rest.Interface
+	InferenceModelRewritesGetter
 	InferenceObjectivesGetter
 	InferencePoolsGetter
 }
@@ -37,6 +38,10 @@ type XInferenceV1alpha2Client struct {
 	restClient rest.Interface
 }
 
+func (c *XInferenceV1alpha2Client) InferenceModelRewrites(namespace string) InferenceModelRewriteInterface {
+	return newInferenceModelRewrites(c, namespace)
+}
+
 func (c *XInferenceV1alpha2Client) InferenceObjectives(namespace string) InferenceObjectiveInterface {
 	return newInferenceObjectives(c, namespace)
 }
diff --git a/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_apix_client.go b/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_apix_client.go
index 54a5b04..c0b2b20 100644
--- a/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_apix_client.go
+++ b/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_apix_client.go
@@ -28,6 +28,10 @@ type FakeXInferenceV1alpha2 struct {
 	*testing.Fake
 }
 
+func (c *FakeXInferenceV1alpha2) InferenceModelRewrites(namespace string) v1alpha2.InferenceModelRewriteInterface {
+	return newFakeInferenceModelRewrites(c, namespace)
+}
+
 func (c *FakeXInferenceV1alpha2) InferenceObjectives(namespace string) v1alpha2.InferenceObjectiveInterface {
 	return newFakeInferenceObjectives(c, namespace)
 }
diff --git a/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_inferencemodelrewrite.go b/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_inferencemodelrewrite.go
new file mode 100644
index 0000000..43f93c8
--- /dev/null
+++ b/client-go/clientset/versioned/typed/apix/v1alpha2/fake/fake_inferencemodelrewrite.go
@@ -0,0 +1,53 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by client-gen. DO NOT EDIT.
+
+package fake
+
+import (
+	gentype "k8s.io/client-go/gentype"
+	v1alpha2 "sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
+	apixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/client-go/applyconfiguration/apix/v1alpha2"
+	typedapixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned/typed/apix/v1alpha2"
+)
+
+// fakeInferenceModelRewrites implements InferenceModelRewriteInterface
+type fakeInferenceModelRewrites struct {
+	*gentype.FakeClientWithListAndApply[*v1alpha2.InferenceModelRewrite, *v1alpha2.InferenceModelRewriteList, *apixv1alpha2.InferenceModelRewriteApplyConfiguration]
+	Fake *FakeXInferenceV1alpha2
+}
+
+func newFakeInferenceModelRewrites(fake *FakeXInferenceV1alpha2, namespace string) typedapixv1alpha2.InferenceModelRewriteInterface {
+	return &fakeInferenceModelRewrites{
+		gentype.NewFakeClientWithListAndApply[*v1alpha2.InferenceModelRewrite, *v1alpha2.InferenceModelRewriteList, *apixv1alpha2.InferenceModelRewriteApplyConfiguration](
+			fake.Fake,
+			namespace,
+			v1alpha2.SchemeGroupVersion.WithResource("inferencemodelrewrites"),
+			v1alpha2.SchemeGroupVersion.WithKind("InferenceModelRewrite"),
+			func() *v1alpha2.InferenceModelRewrite { return &v1alpha2.InferenceModelRewrite{} },
+			func() *v1alpha2.InferenceModelRewriteList { return &v1alpha2.InferenceModelRewriteList{} },
+			func(dst, src *v1alpha2.InferenceModelRewriteList) { dst.ListMeta = src.ListMeta },
+			func(list *v1alpha2.InferenceModelRewriteList) []*v1alpha2.InferenceModelRewrite {
+				return gentype.ToPointerSlice(list.Items)
+			},
+			func(list *v1alpha2.InferenceModelRewriteList, items []*v1alpha2.InferenceModelRewrite) {
+				list.Items = gentype.FromPointerSlice(items)
+			},
+		),
+		fake,
+	}
+}
diff --git a/client-go/clientset/versioned/typed/apix/v1alpha2/generated_expansion.go b/client-go/clientset/versioned/typed/apix/v1alpha2/generated_expansion.go
index b85a89d..dfd94b6 100644
--- a/client-go/clientset/versioned/typed/apix/v1alpha2/generated_expansion.go
+++ b/client-go/clientset/versioned/typed/apix/v1alpha2/generated_expansion.go
@@ -18,6 +18,8 @@ limitations under the License.
 
 package v1alpha2
 
+type InferenceModelRewriteExpansion interface{}
+
 type InferenceObjectiveExpansion interface{}
 
 type InferencePoolExpansion interface{}
diff --git a/client-go/clientset/versioned/typed/apix/v1alpha2/inferencemodelrewrite.go b/client-go/clientset/versioned/typed/apix/v1alpha2/inferencemodelrewrite.go
new file mode 100644
index 0000000..0fac361
--- /dev/null
+++ b/client-go/clientset/versioned/typed/apix/v1alpha2/inferencemodelrewrite.go
@@ -0,0 +1,74 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by client-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	context "context"
+
+	v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	types "k8s.io/apimachinery/pkg/types"
+	watch "k8s.io/apimachinery/pkg/watch"
+	gentype "k8s.io/client-go/gentype"
+	apixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
+	applyconfigurationapixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/client-go/applyconfiguration/apix/v1alpha2"
+	scheme "sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned/scheme"
+)
+
+// InferenceModelRewritesGetter has a method to return a InferenceModelRewriteInterface.
+// A group's client should implement this interface.
+type InferenceModelRewritesGetter interface {
+	InferenceModelRewrites(namespace string) InferenceModelRewriteInterface
+}
+
+// InferenceModelRewriteInterface has methods to work with InferenceModelRewrite resources.
+type InferenceModelRewriteInterface interface {
+	Create(ctx context.Context, inferenceModelRewrite *apixv1alpha2.InferenceModelRewrite, opts v1.CreateOptions) (*apixv1alpha2.InferenceModelRewrite, error)
+	Update(ctx context.Context, inferenceModelRewrite *apixv1alpha2.InferenceModelRewrite, opts v1.UpdateOptions) (*apixv1alpha2.InferenceModelRewrite, error)
+	// Add a +genclient:noStatus comment above the type to avoid generating UpdateStatus().
+	UpdateStatus(ctx context.Context, inferenceModelRewrite *apixv1alpha2.InferenceModelRewrite, opts v1.UpdateOptions) (*apixv1alpha2.InferenceModelRewrite, error)
+	Delete(ctx context.Context, name string, opts v1.DeleteOptions) error
+	DeleteCollection(ctx context.Context, opts v1.DeleteOptions, listOpts v1.ListOptions) error
+	Get(ctx context.Context, name string, opts v1.GetOptions) (*apixv1alpha2.InferenceModelRewrite, error)
+	List(ctx context.Context, opts v1.ListOptions) (*apixv1alpha2.InferenceModelRewriteList, error)
+	Watch(ctx context.Context, opts v1.ListOptions) (watch.Interface, error)
+	Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts v1.PatchOptions, subresources ...string) (result *apixv1alpha2.InferenceModelRewrite, err error)
+	Apply(ctx context.Context, inferenceModelRewrite *applyconfigurationapixv1alpha2.InferenceModelRewriteApplyConfiguration, opts v1.ApplyOptions) (result *apixv1alpha2.InferenceModelRewrite, err error)
+	// Add a +genclient:noStatus comment above the type to avoid generating ApplyStatus().
+	ApplyStatus(ctx context.Context, inferenceModelRewrite *applyconfigurationapixv1alpha2.InferenceModelRewriteApplyConfiguration, opts v1.ApplyOptions) (result *apixv1alpha2.InferenceModelRewrite, err error)
+	InferenceModelRewriteExpansion
+}
+
+// inferenceModelRewrites implements InferenceModelRewriteInterface
+type inferenceModelRewrites struct {
+	*gentype.ClientWithListAndApply[*apixv1alpha2.InferenceModelRewrite, *apixv1alpha2.InferenceModelRewriteList, *applyconfigurationapixv1alpha2.InferenceModelRewriteApplyConfiguration]
+}
+
+// newInferenceModelRewrites returns a InferenceModelRewrites
+func newInferenceModelRewrites(c *XInferenceV1alpha2Client, namespace string) *inferenceModelRewrites {
+	return &inferenceModelRewrites{
+		gentype.NewClientWithListAndApply[*apixv1alpha2.InferenceModelRewrite, *apixv1alpha2.InferenceModelRewriteList, *applyconfigurationapixv1alpha2.InferenceModelRewriteApplyConfiguration](
+			"inferencemodelrewrites",
+			c.RESTClient(),
+			scheme.ParameterCodec,
+			namespace,
+			func() *apixv1alpha2.InferenceModelRewrite { return &apixv1alpha2.InferenceModelRewrite{} },
+			func() *apixv1alpha2.InferenceModelRewriteList { return &apixv1alpha2.InferenceModelRewriteList{} },
+		),
+	}
+}
diff --git a/client-go/informers/externalversions/apix/v1alpha2/inferencemodelrewrite.go b/client-go/informers/externalversions/apix/v1alpha2/inferencemodelrewrite.go
new file mode 100644
index 0000000..c5f0b97
--- /dev/null
+++ b/client-go/informers/externalversions/apix/v1alpha2/inferencemodelrewrite.go
@@ -0,0 +1,102 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by informer-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	context "context"
+	time "time"
+
+	v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	runtime "k8s.io/apimachinery/pkg/runtime"
+	watch "k8s.io/apimachinery/pkg/watch"
+	cache "k8s.io/client-go/tools/cache"
+	gatewayapiinferenceextensionapixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
+	versioned "sigs.k8s.io/gateway-api-inference-extension/client-go/clientset/versioned"
+	internalinterfaces "sigs.k8s.io/gateway-api-inference-extension/client-go/informers/externalversions/internalinterfaces"
+	apixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/client-go/listers/apix/v1alpha2"
+)
+
+// InferenceModelRewriteInformer provides access to a shared informer and lister for
+// InferenceModelRewrites.
+type InferenceModelRewriteInformer interface {
+	Informer() cache.SharedIndexInformer
+	Lister() apixv1alpha2.InferenceModelRewriteLister
+}
+
+type inferenceModelRewriteInformer struct {
+	factory          internalinterfaces.SharedInformerFactory
+	tweakListOptions internalinterfaces.TweakListOptionsFunc
+	namespace        string
+}
+
+// NewInferenceModelRewriteInformer constructs a new informer for InferenceModelRewrite type.
+// Always prefer using an informer factory to get a shared informer instead of getting an independent
+// one. This reduces memory footprint and number of connections to the server.
+func NewInferenceModelRewriteInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers) cache.SharedIndexInformer {
+	return NewFilteredInferenceModelRewriteInformer(client, namespace, resyncPeriod, indexers, nil)
+}
+
+// NewFilteredInferenceModelRewriteInformer constructs a new informer for InferenceModelRewrite type.
+// Always prefer using an informer factory to get a shared informer instead of getting an independent
+// one. This reduces memory footprint and number of connections to the server.
+func NewFilteredInferenceModelRewriteInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers, tweakListOptions internalinterfaces.TweakListOptionsFunc) cache.SharedIndexInformer {
+	return cache.NewSharedIndexInformer(
+		&cache.ListWatch{
+			ListFunc: func(options v1.ListOptions) (runtime.Object, error) {
+				if tweakListOptions != nil {
+					tweakListOptions(&options)
+				}
+				return client.XInferenceV1alpha2().InferenceModelRewrites(namespace).List(context.Background(), options)
+			},
+			WatchFunc: func(options v1.ListOptions) (watch.Interface, error) {
+				if tweakListOptions != nil {
+					tweakListOptions(&options)
+				}
+				return client.XInferenceV1alpha2().InferenceModelRewrites(namespace).Watch(context.Background(), options)
+			},
+			ListWithContextFunc: func(ctx context.Context, options v1.ListOptions) (runtime.Object, error) {
+				if tweakListOptions != nil {
+					tweakListOptions(&options)
+				}
+				return client.XInferenceV1alpha2().InferenceModelRewrites(namespace).List(ctx, options)
+			},
+			WatchFuncWithContext: func(ctx context.Context, options v1.ListOptions) (watch.Interface, error) {
+				if tweakListOptions != nil {
+					tweakListOptions(&options)
+				}
+				return client.XInferenceV1alpha2().InferenceModelRewrites(namespace).Watch(ctx, options)
+			},
+		},
+		&gatewayapiinferenceextensionapixv1alpha2.InferenceModelRewrite{},
+		resyncPeriod,
+		indexers,
+	)
+}
+
+func (f *inferenceModelRewriteInformer) defaultInformer(client versioned.Interface, resyncPeriod time.Duration) cache.SharedIndexInformer {
+	return NewFilteredInferenceModelRewriteInformer(client, f.namespace, resyncPeriod, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, f.tweakListOptions)
+}
+
+func (f *inferenceModelRewriteInformer) Informer() cache.SharedIndexInformer {
+	return f.factory.InformerFor(&gatewayapiinferenceextensionapixv1alpha2.InferenceModelRewrite{}, f.defaultInformer)
+}
+
+func (f *inferenceModelRewriteInformer) Lister() apixv1alpha2.InferenceModelRewriteLister {
+	return apixv1alpha2.NewInferenceModelRewriteLister(f.Informer().GetIndexer())
+}
diff --git a/client-go/informers/externalversions/apix/v1alpha2/interface.go b/client-go/informers/externalversions/apix/v1alpha2/interface.go
index 9f69816..11ccf5c 100644
--- a/client-go/informers/externalversions/apix/v1alpha2/interface.go
+++ b/client-go/informers/externalversions/apix/v1alpha2/interface.go
@@ -24,6 +24,8 @@ import (
 
 // Interface provides access to all the informers in this group version.
 type Interface interface {
+	// InferenceModelRewrites returns a InferenceModelRewriteInformer.
+	InferenceModelRewrites() InferenceModelRewriteInformer
 	// InferenceObjectives returns a InferenceObjectiveInformer.
 	InferenceObjectives() InferenceObjectiveInformer
 	// InferencePools returns a InferencePoolInformer.
@@ -41,6 +43,11 @@ func New(f internalinterfaces.SharedInformerFactory, namespace string, tweakList
 	return &version{factory: f, namespace: namespace, tweakListOptions: tweakListOptions}
 }
 
+// InferenceModelRewrites returns a InferenceModelRewriteInformer.
+func (v *version) InferenceModelRewrites() InferenceModelRewriteInformer {
+	return &inferenceModelRewriteInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions}
+}
+
 // InferenceObjectives returns a InferenceObjectiveInformer.
 func (v *version) InferenceObjectives() InferenceObjectiveInformer {
 	return &inferenceObjectiveInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions}
diff --git a/client-go/informers/externalversions/generic.go b/client-go/informers/externalversions/generic.go
index 2fe2915..5eeaea4 100644
--- a/client-go/informers/externalversions/generic.go
+++ b/client-go/informers/externalversions/generic.go
@@ -63,6 +63,8 @@ func (f *sharedInformerFactory) ForResource(resource schema.GroupVersionResource
 		return &genericInformer{resource: resource.GroupResource(), informer: f.XInference().V1alpha1().InferencePoolImports().Informer()}, nil
 
 		// Group=inference.networking.x-k8s.io, Version=v1alpha2
+	case v1alpha2.SchemeGroupVersion.WithResource("inferencemodelrewrites"):
+		return &genericInformer{resource: resource.GroupResource(), informer: f.XInference().V1alpha2().InferenceModelRewrites().Informer()}, nil
 	case v1alpha2.SchemeGroupVersion.WithResource("inferenceobjectives"):
 		return &genericInformer{resource: resource.GroupResource(), informer: f.XInference().V1alpha2().InferenceObjectives().Informer()}, nil
 	case v1alpha2.SchemeGroupVersion.WithResource("inferencepools"):
diff --git a/client-go/listers/apix/v1alpha2/expansion_generated.go b/client-go/listers/apix/v1alpha2/expansion_generated.go
index 09f6d1d..b20936e 100644
--- a/client-go/listers/apix/v1alpha2/expansion_generated.go
+++ b/client-go/listers/apix/v1alpha2/expansion_generated.go
@@ -18,6 +18,14 @@ limitations under the License.
 
 package v1alpha2
 
+// InferenceModelRewriteListerExpansion allows custom methods to be added to
+// InferenceModelRewriteLister.
+type InferenceModelRewriteListerExpansion interface{}
+
+// InferenceModelRewriteNamespaceListerExpansion allows custom methods to be added to
+// InferenceModelRewriteNamespaceLister.
+type InferenceModelRewriteNamespaceListerExpansion interface{}
+
 // InferenceObjectiveListerExpansion allows custom methods to be added to
 // InferenceObjectiveLister.
 type InferenceObjectiveListerExpansion interface{}
diff --git a/client-go/listers/apix/v1alpha2/inferencemodelrewrite.go b/client-go/listers/apix/v1alpha2/inferencemodelrewrite.go
new file mode 100644
index 0000000..8209c29
--- /dev/null
+++ b/client-go/listers/apix/v1alpha2/inferencemodelrewrite.go
@@ -0,0 +1,70 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by lister-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	labels "k8s.io/apimachinery/pkg/labels"
+	listers "k8s.io/client-go/listers"
+	cache "k8s.io/client-go/tools/cache"
+	apixv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
+)
+
+// InferenceModelRewriteLister helps list InferenceModelRewrites.
+// All objects returned here must be treated as read-only.
+type InferenceModelRewriteLister interface {
+	// List lists all InferenceModelRewrites in the indexer.
+	// Objects returned here must be treated as read-only.
+	List(selector labels.Selector) (ret []*apixv1alpha2.InferenceModelRewrite, err error)
+	// InferenceModelRewrites returns an object that can list and get InferenceModelRewrites.
+	InferenceModelRewrites(namespace string) InferenceModelRewriteNamespaceLister
+	InferenceModelRewriteListerExpansion
+}
+
+// inferenceModelRewriteLister implements the InferenceModelRewriteLister interface.
+type inferenceModelRewriteLister struct {
+	listers.ResourceIndexer[*apixv1alpha2.InferenceModelRewrite]
+}
+
+// NewInferenceModelRewriteLister returns a new InferenceModelRewriteLister.
+func NewInferenceModelRewriteLister(indexer cache.Indexer) InferenceModelRewriteLister {
+	return &inferenceModelRewriteLister{listers.New[*apixv1alpha2.InferenceModelRewrite](indexer, apixv1alpha2.Resource("inferencemodelrewrite"))}
+}
+
+// InferenceModelRewrites returns an object that can list and get InferenceModelRewrites.
+func (s *inferenceModelRewriteLister) InferenceModelRewrites(namespace string) InferenceModelRewriteNamespaceLister {
+	return inferenceModelRewriteNamespaceLister{listers.NewNamespaced[*apixv1alpha2.InferenceModelRewrite](s.ResourceIndexer, namespace)}
+}
+
+// InferenceModelRewriteNamespaceLister helps list and get InferenceModelRewrites.
+// All objects returned here must be treated as read-only.
+type InferenceModelRewriteNamespaceLister interface {
+	// List lists all InferenceModelRewrites in the indexer for a given namespace.
+	// Objects returned here must be treated as read-only.
+	List(selector labels.Selector) (ret []*apixv1alpha2.InferenceModelRewrite, err error)
+	// Get retrieves the InferenceModelRewrite from the indexer for a given namespace and name.
+	// Objects returned here must be treated as read-only.
+	Get(name string) (*apixv1alpha2.InferenceModelRewrite, error)
+	InferenceModelRewriteNamespaceListerExpansion
+}
+
+// inferenceModelRewriteNamespaceLister implements the InferenceModelRewriteNamespaceLister
+// interface.
+type inferenceModelRewriteNamespaceLister struct {
+	listers.ResourceIndexer[*apixv1alpha2.InferenceModelRewrite]
+}
diff --git a/cmd/bbr/main.go b/cmd/bbr/main.go
index d79d7c6..ffa1c5f 100644
--- a/cmd/bbr/main.go
+++ b/cmd/bbr/main.go
@@ -17,141 +17,15 @@ limitations under the License.
 package main
 
 import (
-	"flag"
-	"fmt"
 	"os"
 
-	"github.com/go-logr/logr"
-	uberzap "go.uber.org/zap"
-	"go.uber.org/zap/zapcore"
-	"google.golang.org/grpc"
-	healthPb "google.golang.org/grpc/health/grpc_health_v1"
 	ctrl "sigs.k8s.io/controller-runtime"
-	"sigs.k8s.io/controller-runtime/pkg/log/zap"
-	"sigs.k8s.io/controller-runtime/pkg/manager"
-	"sigs.k8s.io/controller-runtime/pkg/metrics/filters"
-	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
-
-	"sigs.k8s.io/gateway-api-inference-extension/internal/runnable"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/metrics"
-	runserver "sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/server"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
-)
-
-var (
-	grpcPort = flag.Int(
-		"grpc-port",
-		9004,
-		"The gRPC port used for communicating with Envoy proxy")
-	grpcHealthPort = flag.Int(
-		"grpc-health-port",
-		9005,
-		"The port used for gRPC liveness and readiness probes")
-	metricsPort = flag.Int(
-		"metrics-port", 9090, "The metrics port")
-	streaming = flag.Bool(
-		"streaming", false, "Enables streaming support for Envoy full-duplex streaming mode")
-	logVerbosity = flag.Int("v", logging.DEFAULT, "number for the log level verbosity")
-
-	setupLog = ctrl.Log.WithName("setup")
+	"sigs.k8s.io/gateway-api-inference-extension/cmd/bbr/runner"
 )
 
 func main() {
-	if err := run(); err != nil {
+	if err := runner.NewRunner().Run(ctrl.SetupSignalHandler()); err != nil {
 		os.Exit(1)
 	}
-}
-
-func run() error {
-	opts := zap.Options{Development: true}
-	opts.BindFlags(flag.CommandLine)
-	flag.Parse()
-	initLogging(&opts)
-
-	// Print all flag values
-	flags := make(map[string]any)
-	flag.VisitAll(func(f *flag.Flag) {
-		flags[f.Name] = f.Value
-	})
-	setupLog.Info("Flags processed", "flags", flags)
-
-	// Init runtime.
-	cfg, err := ctrl.GetConfig()
-	if err != nil {
-		setupLog.Error(err, "Failed to get rest config")
-		return err
-	}
-
-	metrics.Register()
-
-	// Register metrics handler.
-	// Metrics endpoint is enabled in 'config/default/kustomization.yaml'. The Metrics options configure the server.
-	// More info:
-	// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.19.1/pkg/metrics/server
-	// - https://book.kubebuilder.io/reference/metrics.html
-	metricsServerOptions := metricsserver.Options{
-		BindAddress:    fmt.Sprintf(":%d", *metricsPort),
-		FilterProvider: filters.WithAuthenticationAndAuthorization,
-	}
-	mgr, err := ctrl.NewManager(cfg, ctrl.Options{Metrics: metricsServerOptions})
-	if err != nil {
-		setupLog.Error(err, "Failed to create manager", "config", cfg)
-		return err
-	}
-
-	ctx := ctrl.SetupSignalHandler()
-
-	// Setup runner.
-	serverRunner := runserver.NewDefaultExtProcServerRunner(*grpcPort, *streaming)
-
-	// Register health server.
-	if err := registerHealthServer(mgr, ctrl.Log.WithName("health"), *grpcHealthPort); err != nil {
-		return err
-	}
-
-	// Register ext-proc server.
-	if err := mgr.Add(serverRunner.AsRunnable(ctrl.Log.WithName("ext-proc"))); err != nil {
-		setupLog.Error(err, "Failed to register ext-proc gRPC server")
-		return err
-	}
-
-	// Start the manager. This blocks until a signal is received.
-	setupLog.Info("Manager starting")
-	if err := mgr.Start(ctx); err != nil {
-		setupLog.Error(err, "Error starting manager")
-		return err
-	}
-	setupLog.Info("Manager terminated")
-	return nil
-}
-
-// registerHealthServer adds the Health gRPC server as a Runnable to the given manager.
-func registerHealthServer(mgr manager.Manager, logger logr.Logger, port int) error {
-	srv := grpc.NewServer()
-	healthPb.RegisterHealthServer(srv, &healthServer{
-		logger: logger,
-	})
-	if err := mgr.Add(
-		runnable.NoLeaderElection(runnable.GRPCServer("health", srv, port))); err != nil {
-		setupLog.Error(err, "Failed to register health server")
-		return err
-	}
-	return nil
-}
-
-func initLogging(opts *zap.Options) {
-	useV := true
-	flag.Visit(func(f *flag.Flag) {
-		if f.Name == "zap-log-level" {
-			useV = false
-		}
-	})
-	if useV {
-		// See https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/log/zap#Options.Level
-		lvl := -1 * (*logVerbosity)
-		opts.Level = uberzap.NewAtomicLevelAt(zapcore.Level(int8(lvl)))
-	}
 
-	logger := zap.New(zap.UseFlagOptions(opts), zap.RawZapOpts(uberzap.AddCaller()))
-	ctrl.SetLogger(logger)
 }
diff --git a/cmd/bbr/health.go b/cmd/bbr/runner/health.go
similarity index 92%
rename from cmd/bbr/health.go
rename to cmd/bbr/runner/health.go
index e1020b3..8000ab8 100644
--- a/cmd/bbr/health.go
+++ b/cmd/bbr/runner/health.go
@@ -14,23 +14,21 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package main
+package runner
 
 import (
 	"context"
 
 	extProcPb "github.com/envoyproxy/go-control-plane/envoy/service/ext_proc/v3"
-	"github.com/go-logr/logr"
 	"google.golang.org/grpc/codes"
 	healthPb "google.golang.org/grpc/health/grpc_health_v1"
 	"google.golang.org/grpc/status"
 
+	"sigs.k8s.io/controller-runtime/pkg/log"
 	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
 )
 
-type healthServer struct {
-	logger logr.Logger
-}
+type healthServer struct{}
 
 func (s *healthServer) Check(ctx context.Context, in *healthPb.HealthCheckRequest) (*healthPb.HealthCheckResponse, error) {
 	// TODO: we're accepting ANY service name for now as a temporary hack in alignment with
@@ -40,7 +38,7 @@ func (s *healthServer) Check(ctx context.Context, in *healthPb.HealthCheckReques
 	// 	return &healthPb.HealthCheckResponse{Status: healthPb.HealthCheckResponse_SERVICE_UNKNOWN}, nil
 	// }
 
-	s.logger.V(logutil.VERBOSE).Info("gRPC health check serving", "service", in.Service)
+	log.FromContext(ctx).V(logutil.DEBUG).Info("gRPC health check serving", "service", in.Service)
 	return &healthPb.HealthCheckResponse{Status: healthPb.HealthCheckResponse_SERVING}, nil
 }
 
diff --git a/cmd/bbr/runner/runner.go b/cmd/bbr/runner/runner.go
new file mode 100644
index 0000000..8a4b429
--- /dev/null
+++ b/cmd/bbr/runner/runner.go
@@ -0,0 +1,156 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package runner
+
+import (
+	"context"
+	"flag"
+	"fmt"
+
+	uberzap "go.uber.org/zap"
+	"go.uber.org/zap/zapcore"
+	"google.golang.org/grpc"
+	healthPb "google.golang.org/grpc/health/grpc_health_v1"
+	ctrl "sigs.k8s.io/controller-runtime"
+	"sigs.k8s.io/controller-runtime/pkg/log/zap"
+	"sigs.k8s.io/controller-runtime/pkg/manager"
+	"sigs.k8s.io/controller-runtime/pkg/metrics/filters"
+	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
+
+	"sigs.k8s.io/gateway-api-inference-extension/internal/runnable"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/metrics"
+	runserver "sigs.k8s.io/gateway-api-inference-extension/pkg/bbr/server"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
+)
+
+var (
+	grpcPort       = flag.Int("grpc-port", 9004, "The gRPC port used for communicating with Envoy proxy")
+	grpcHealthPort = flag.Int("grpc-health-port", 9005, "The port used for gRPC liveness and readiness probes")
+	metricsPort    = flag.Int("metrics-port", 9090, "The metrics port")
+	streaming      = flag.Bool("streaming", false, "Enables streaming support for Envoy full-duplex streaming mode")
+	logVerbosity   = flag.Int("v", logging.DEFAULT, "number for the log level verbosity")
+
+	setupLog = ctrl.Log.WithName("setup")
+)
+
+func NewRunner() *Runner {
+	return &Runner{
+		bbrExecutableName: "BBR",
+	}
+}
+
+// Runner is used to run bbr with its plugins
+type Runner struct {
+	bbrExecutableName string
+}
+
+// WithExecutableName sets the name of the executable containing the runner.
+// The name is used in the version log upon startup and is otherwise opaque.
+func (r *Runner) WithExecutableName(exeName string) *Runner {
+	r.bbrExecutableName = exeName
+	return r
+}
+
+func (r *Runner) Run(ctx context.Context) error {
+	opts := zap.Options{Development: true}
+	opts.BindFlags(flag.CommandLine)
+	flag.Parse()
+	initLogging(&opts)
+
+	// Print all flag values
+	flags := make(map[string]any)
+	flag.VisitAll(func(f *flag.Flag) {
+		flags[f.Name] = f.Value
+	})
+	setupLog.Info("Flags processed", "flags", flags)
+
+	// Init runtime.
+	cfg, err := ctrl.GetConfig()
+	if err != nil {
+		setupLog.Error(err, "Failed to get rest config")
+		return err
+	}
+
+	metrics.Register()
+
+	// Register metrics handler.
+	// Metrics endpoint is enabled in 'config/default/kustomization.yaml'. The Metrics options configure the server.
+	// More info:
+	// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.19.1/pkg/metrics/server
+	// - https://book.kubebuilder.io/reference/metrics.html
+	metricsServerOptions := metricsserver.Options{
+		BindAddress:    fmt.Sprintf(":%d", *metricsPort),
+		FilterProvider: filters.WithAuthenticationAndAuthorization,
+	}
+	mgr, err := ctrl.NewManager(cfg, ctrl.Options{Metrics: metricsServerOptions})
+	if err != nil {
+		setupLog.Error(err, "Failed to create manager", "config", cfg)
+		return err
+	}
+
+	// Setup runner.
+	serverRunner := runserver.NewDefaultExtProcServerRunner(*grpcPort, *streaming)
+
+	// Register health server.
+	if err := registerHealthServer(mgr, *grpcHealthPort); err != nil {
+		return err
+	}
+
+	// Register ext-proc server.
+	if err := mgr.Add(serverRunner.AsRunnable(ctrl.Log.WithName("ext-proc"))); err != nil {
+		setupLog.Error(err, "Failed to register ext-proc gRPC server")
+		return err
+	}
+
+	// Start the manager. This blocks until a signal is received.
+	setupLog.Info("Manager starting")
+	if err := mgr.Start(ctx); err != nil {
+		setupLog.Error(err, "Error starting manager")
+		return err
+	}
+	setupLog.Info("Manager terminated")
+	return nil
+}
+
+// registerHealthServer adds the Health gRPC server as a Runnable to the given manager.
+func registerHealthServer(mgr manager.Manager, port int) error {
+	srv := grpc.NewServer()
+	healthPb.RegisterHealthServer(srv, &healthServer{})
+	if err := mgr.Add(
+		runnable.NoLeaderElection(runnable.GRPCServer("health", srv, port))); err != nil {
+		setupLog.Error(err, "Failed to register health server")
+		return err
+	}
+	return nil
+}
+
+func initLogging(opts *zap.Options) {
+	useV := true
+	flag.Visit(func(f *flag.Flag) {
+		if f.Name == "zap-log-level" {
+			useV = false
+		}
+	})
+	if useV {
+		// See https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/log/zap#Options.Level
+		lvl := -1 * (*logVerbosity)
+		opts.Level = uberzap.NewAtomicLevelAt(zapcore.Level(int8(lvl)))
+	}
+
+	logger := zap.New(zap.UseFlagOptions(opts), zap.RawZapOpts(uberzap.AddCaller()))
+	ctrl.SetLogger(logger)
+}
diff --git a/cmd/epp/runner/runner.go b/cmd/epp/runner/runner.go
index da81fdf..e6caafa 100644
--- a/cmd/epp/runner/runner.go
+++ b/cmd/epp/runner/runner.go
@@ -44,9 +44,11 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/metrics/filters"
 	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
 
+	configapi "sigs.k8s.io/gateway-api-inference-extension/apix/config/v1alpha1"
 	"sigs.k8s.io/gateway-api-inference-extension/internal/runnable"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/common"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/config/loader"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
 	dlmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer/metrics"
@@ -77,6 +79,11 @@ const (
 	// enableExperimentalFlowControlLayer defines the environment variable used as a feature flag for the pluggable flow
 	// control layer.
 	enableExperimentalFlowControlLayer = "ENABLE_EXPERIMENTAL_FLOW_CONTROL_LAYER"
+
+	// Saturation Detector deprecated configuration environment variables
+	EnvSdQueueDepthThreshold       = "SD_QUEUE_DEPTH_THRESHOLD"
+	EnvSdKVCacheUtilThreshold      = "SD_KV_CACHE_UTIL_THRESHOLD"
+	EnvSdMetricsStalenessThreshold = "SD_METRICS_STALENESS_THRESHOLD"
 )
 
 // TODO: this is hardcoded for POC only. This needs to be hooked up to our text-based config story.
@@ -143,6 +150,7 @@ func NewRunner() *Runner {
 // Runner is used to run epp with its plugins
 type Runner struct {
 	eppExecutableName    string // the EPP executable name
+	featureGates         config.FeatureConfig
 	requestControlConfig *requestcontrol.Config
 	schedulerConfig      *scheduling.SchedulerConfig
 	customCollectors     []prometheus.Collector
@@ -200,9 +208,6 @@ func (r *Runner) Run(ctx context.Context) error {
 	})
 	setupLog.Info("Flags processed", "flags", flags)
 
-	// --- Load Configurations from Environment Variables ---
-	sdConfig := saturationdetector.LoadConfigFromEnv()
-
 	// --- Get Kubernetes Config ---
 	cfg, err := ctrl.GetConfig()
 	if err != nil {
@@ -210,14 +215,26 @@ func (r *Runner) Run(ctx context.Context) error {
 		return err
 	}
 
+	rawConfig, featureGates, err := r.parseConfigurationPhaseOne(ctx)
+	if err != nil {
+		setupLog.Error(err, "Failed to parse configuration")
+		return err
+	}
+	r.featureGates = featureGates
+
 	// --- Setup Datastore ---
-	useDatalayerV2 := env.GetEnvBool(enableExperimentalDatalayerV2, false, setupLog)
-	epf, err := r.setupMetricsCollection(setupLog, useDatalayerV2)
+	epf, err := r.setupMetricsCollection(setupLog, r.featureGates[datalayer.FeatureGate])
 	if err != nil {
 		return err
 	}
 	datastore := datastore.NewDatastore(ctx, epf, int32(*modelServerMetricsPort))
 
+	eppConfig, err := r.parseConfigurationPhaseTwo(ctx, rawConfig, datastore)
+	if err != nil {
+		setupLog.Error(err, "Failed to parse configuration")
+		return err
+	}
+
 	// --- Setup Metrics Server ---
 	customCollectors := []prometheus.Collector{collectors.NewInferencePoolMetricsCollector(datastore)}
 	if r.customCollectors != nil {
@@ -297,12 +314,6 @@ func (r *Runner) Run(ctx context.Context) error {
 		runtime.SetBlockProfileRate(1)
 	}
 
-	err = r.parsePluginsConfiguration(ctx, datastore)
-	if err != nil {
-		setupLog.Error(err, "Failed to parse plugins configuration")
-		return err
-	}
-
 	// --- Initialize Core EPP Components ---
 	if r.schedulerConfig == nil {
 		err := errors.New("scheduler config must be set either by config api or through code")
@@ -314,12 +325,11 @@ func (r *Runner) Run(ctx context.Context) error {
 
 	scheduler := scheduling.NewSchedulerWithConfig(r.schedulerConfig)
 
-	saturationDetector := saturationdetector.NewDetector(sdConfig, setupLog)
+	saturationDetector := saturationdetector.NewDetector(eppConfig.SaturationDetectorConfig, setupLog)
 
 	// --- Admission Control Initialization ---
-	enableFlowControl := env.GetEnvBool(enableExperimentalFlowControlLayer, false, setupLog)
 	var admissionController requestcontrol.AdmissionController
-	if enableFlowControl {
+	if r.featureGates[flowcontrol.FeatureGate] {
 		setupLog.Info("Initializing experimental Flow Control layer")
 		fcCfg, err := flowControlConfig.ValidateAndApplyDefaults()
 		if err != nil {
@@ -367,7 +377,7 @@ func (r *Runner) Run(ctx context.Context) error {
 		MetricsStalenessThreshold:        *metricsStalenessThreshold,
 		Director:                         director,
 		SaturationDetector:               saturationDetector,
-		UseExperimentalDatalayerV2:       useDatalayerV2, // pluggable data layer feature flag
+		UseExperimentalDatalayerV2:       r.featureGates[datalayer.FeatureGate], // pluggable data layer feature flag
 	}
 	if err := serverRunner.SetupWithManager(ctx, mgr); err != nil {
 		setupLog.Error(err, "Failed to setup EPP controllers")
@@ -410,9 +420,9 @@ func (r *Runner) registerInTreePlugins() {
 	plugins.Register(testfilter.HeaderBasedTestingFilterType, testfilter.HeaderBasedTestingFilterFactory)
 }
 
-func (r *Runner) parsePluginsConfiguration(ctx context.Context, ds datastore.Datastore) error {
+func (r *Runner) parseConfigurationPhaseOne(ctx context.Context) (*configapi.EndpointPickerConfig, config.FeatureConfig, error) {
 	if *configText == "" && *configFile == "" {
-		return nil // configuring through code, not through file
+		return nil, nil, nil // configuring through code, not through file
 	}
 
 	logger := log.FromContext(ctx)
@@ -424,30 +434,80 @@ func (r *Runner) parsePluginsConfiguration(ctx context.Context, ds datastore.Dat
 		var err error
 		configBytes, err = os.ReadFile(*configFile)
 		if err != nil {
-			return fmt.Errorf("failed to load config from a file '%s' - %w", *configFile, err)
+			return nil, nil, fmt.Errorf("failed to load config from a file '%s' - %w", *configFile, err)
 		}
 	}
 
+	loader.RegisterFeatureGate(datalayer.FeatureGate)
+	loader.RegisterFeatureGate(flowcontrol.FeatureGate)
+
 	r.registerInTreePlugins()
+
+	return loader.LoadConfigPhaseOne(configBytes, logger)
+}
+
+func (r *Runner) parseConfigurationPhaseTwo(ctx context.Context, rawConfig *configapi.EndpointPickerConfig, ds datastore.Datastore) (*config.Config, error) {
+	logger := log.FromContext(ctx)
 	handle := plugins.NewEppHandle(ctx, ds.PodList)
-	config, err := loader.LoadConfig(configBytes, handle, logger)
+	cfg, err := loader.LoadConfigPhaseTwo(rawConfig, handle, logger)
 
 	if err != nil {
-		return fmt.Errorf("failed to load the configuration - %w", err)
+		return nil, fmt.Errorf("failed to load the configuration - %w", err)
 	}
 
-	r.schedulerConfig = config.SchedulerConfig
+	r.schedulerConfig = cfg.SchedulerConfig
 
 	// Add requestControl plugins
 	r.requestControlConfig.AddPlugins(handle.GetAllPlugins()...)
 
+	// Handler deprecated configuration options
+	r.deprecatedConfigurationHelper(cfg, logger)
+
 	logger.Info("loaded configuration from file/text successfully")
-	return nil
+	return cfg, nil
+}
+
+func (r *Runner) deprecatedConfigurationHelper(cfg *config.Config, logger logr.Logger) {
+	// Handle deprecated environment variable based feature flags
+
+	if _, ok := os.LookupEnv(enableExperimentalDatalayerV2); ok {
+		logger.Info("Enabling the experimental Data Layer V2 using environment variables is deprecated")
+		r.featureGates[datalayer.FeatureGate] = env.GetEnvBool(enableExperimentalDatalayerV2, false, logger)
+	}
+	if _, ok := os.LookupEnv(enableExperimentalFlowControlLayer); ok {
+		logger.Info("Enabling the experimental Flow Control layer using environment variables is deprecated")
+		r.featureGates[flowcontrol.FeatureGate] = env.GetEnvBool(enableExperimentalFlowControlLayer, false, setupLog)
+	}
+
+	// Handle deprecated environment variable base Saturation Detector configuration
+
+	if _, ok := os.LookupEnv(EnvSdQueueDepthThreshold); ok {
+		logger.Info("Configuring Saturation Detector using environment variables is deprecated")
+		cfg.SaturationDetectorConfig.QueueDepthThreshold =
+			env.GetEnvInt(EnvSdQueueDepthThreshold, saturationdetector.DefaultQueueDepthThreshold, logger)
+		if cfg.SaturationDetectorConfig.QueueDepthThreshold <= 0 {
+			cfg.SaturationDetectorConfig.QueueDepthThreshold = saturationdetector.DefaultQueueDepthThreshold
+		}
+	}
+	if _, ok := os.LookupEnv(EnvSdKVCacheUtilThreshold); ok {
+		logger.Info("Configuring Saturation Detector using environment variables is deprecated")
+		cfg.SaturationDetectorConfig.KVCacheUtilThreshold = env.GetEnvFloat(EnvSdKVCacheUtilThreshold, saturationdetector.DefaultKVCacheUtilThreshold, logger)
+		if cfg.SaturationDetectorConfig.KVCacheUtilThreshold <= 0 || cfg.SaturationDetectorConfig.KVCacheUtilThreshold >= 1 {
+			cfg.SaturationDetectorConfig.KVCacheUtilThreshold = saturationdetector.DefaultKVCacheUtilThreshold
+		}
+	}
+	if _, ok := os.LookupEnv(EnvSdMetricsStalenessThreshold); ok {
+		logger.Info("Configuring Saturation Detector using environment variables is deprecated")
+		cfg.SaturationDetectorConfig.MetricsStalenessThreshold = env.GetEnvDuration(EnvSdMetricsStalenessThreshold, saturationdetector.DefaultMetricsStalenessThreshold, logger)
+		if cfg.SaturationDetectorConfig.MetricsStalenessThreshold <= 0 {
+			cfg.SaturationDetectorConfig.MetricsStalenessThreshold = saturationdetector.DefaultMetricsStalenessThreshold
+		}
+	}
 }
 
 func (r *Runner) setupMetricsCollection(setupLog logr.Logger, useExperimentalDatalayer bool) (datalayer.EndpointFactory, error) {
 	if useExperimentalDatalayer {
-		return setupDatalayer()
+		return setupDatalayer(setupLog)
 	}
 
 	if len(datalayer.GetSources()) != 0 {
@@ -492,11 +552,16 @@ func setupMetricsV1(setupLog logr.Logger) (datalayer.EndpointFactory, error) {
 	return pmf, nil
 }
 
-func setupDatalayer() (datalayer.EndpointFactory, error) {
-	// create and register a metrics data source and extractor. In the future,
-	// data sources and extractors might be configured via a file. Once done,
-	// this (and registering the sources with the endpoint factory) should
-	// be moved accordingly.
+// This function serves two (independent) purposes:
+// - creating data sources and configuring their extractors.
+// - configuring endpoint factory with the provided source.
+// In the future, data sources and extractors might be configured via
+// a file. Once done, this (and registering the sources with the
+// endpoint factory) should be moved accordingly.
+// Regardless, registration of all sources (e.g., if additional sources
+// are to be configured), must be done before the EndpointFactory is initialized.
+func setupDatalayer(logger logr.Logger) (datalayer.EndpointFactory, error) {
+	// create and register a metrics data source and extractor.
 	source := dlmetrics.NewDataSource(*modelServerMetricsScheme,
 		*modelServerMetricsPath,
 		*modelServerMetricsHttpsInsecureSkipVerify,
@@ -515,7 +580,12 @@ func setupDatalayer() (datalayer.EndpointFactory, error) {
 		return nil, err
 	}
 
-	factory := datalayer.NewEndpointFactory(datalayer.GetSources(), *refreshMetricsInterval)
+	// TODO: this could be moved to the configuration loading functions once ported over.
+	sources := datalayer.GetSources()
+	for _, src := range sources {
+		logger.Info("data layer configuration", "source", src.Name(), "extractors", src.Extractors())
+	}
+	factory := datalayer.NewEndpointFactory(sources, *refreshMetricsInterval)
 	return factory, nil
 }
 
diff --git a/config/charts/inferencepool/templates/epp-config.yaml b/config/charts/inferencepool/templates/epp-config.yaml
index ca1bda5..31b9e53 100644
--- a/config/charts/inferencepool/templates/epp-config.yaml
+++ b/config/charts/inferencepool/templates/epp-config.yaml
@@ -24,3 +24,13 @@ data:
   {{- .Values.inferenceExtension.pluginsCustomConfig | toYaml | nindent 2 }}
   {{- end }}
   
+---
+{{- if .Values.inferenceExtension.sidecar.enabled }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ .Values.inferenceExtension.sidecar.configMap.name }}
+  namespace: {{ .Release.Namespace }}
+data:
+  {{- .Values.inferenceExtension.sidecar.configMap.data | toYaml | nindent 2 }}
+{{- end }}
diff --git a/config/charts/inferencepool/templates/epp-deployment.yaml b/config/charts/inferencepool/templates/epp-deployment.yaml
index e567701..5b3634c 100644
--- a/config/charts/inferencepool/templates/epp-deployment.yaml
+++ b/config/charts/inferencepool/templates/epp-deployment.yaml
@@ -27,9 +27,48 @@ spec:
       # Conservatively, this timeout should mirror the longest grace period of the pods within the pool
       terminationGracePeriodSeconds: 130
       containers:
+      {{- if .Values.inferenceExtension.sidecar.enabled }}
+      - name: {{ .Values.inferenceExtension.sidecar.name }}
+        image: {{ .Values.inferenceExtension.sidecar.image }}
+        imagePullPolicy: {{ .Values.inferenceExtension.sidecar.image.pullPolicy | default "IfNotPresent" }}
+        {{- with .Values.inferenceExtension.sidecar.command }}
+        command:
+          - {{ . | quote }}
+        {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.args }}
+        args:
+          {{- range . }}
+            - {{ tpl . $ | quote }}
+          {{- end }}
+        {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.env }}
+        env:
+          {{- toYaml . | nindent 10 }}
+          {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.ports }}
+        ports:
+          {{- toYaml . | nindent 10 }}
+        {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.livenessProbe }}
+        livenessProbe:
+          {{- toYaml . | nindent 10 }}
+        {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.readinessProbe }}
+        readinessProbe:
+          {{- toYaml . | nindent 10 }}
+        {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.resources }}
+        resources:
+          {{- toYaml . | nindent 10 }}
+        {{- end }}
+        {{- with .Values.inferenceExtension.sidecar.volumeMounts }}
+        volumeMounts:
+          {{- toYaml . | nindent 10 }}
+        {{- end }}
+        {{- end }}
       - name: epp
         image: {{ .Values.inferenceExtension.image.hub }}/{{ .Values.inferenceExtension.image.name }}:{{ .Values.inferenceExtension.image.tag }}
-        imagePullPolicy: {{ .Values.inferenceExtension.image.pullPolicy | default "Always" }}
+        imagePullPolicy: {{ .Values.inferenceExtension.image.pullPolicy | default "IfNotPresent" }}
         args:
         - --pool-name
         - {{ .Release.Name }}
@@ -139,6 +178,9 @@ spec:
         - name: plugins-config-volume
           mountPath: "/config"
       volumes:
+      {{- if .Values.inferenceExtension.sidecar.volumes }}
+      {{- tpl (toYaml .Values.inferenceExtension.sidecar.volumes) $ | nindent 6 }}
+      {{- end }}
       - name: plugins-config-volume
         configMap:
           name: {{ include "gateway-api-inference-extension.name" . }}
diff --git a/config/charts/inferencepool/templates/istio.yaml b/config/charts/inferencepool/templates/istio.yaml
index b50c0b0..a0f9c8d 100644
--- a/config/charts/inferencepool/templates/istio.yaml
+++ b/config/charts/inferencepool/templates/istio.yaml
@@ -1,16 +1,22 @@
 {{- if eq .Values.provider.name "istio" }}
+{{- /* Prefer .Values.provider.istio, fallback to legacy .Values.istio, then {} */ -}}
+{{- $provIstio := (index .Values "provider" "istio") -}}
+{{- $legacyIstio := .Values.istio -}}
+{{- $istio := coalesce $provIstio $legacyIstio (dict) -}}
+{{- $dr := (index $istio "destinationRule") | default (dict) -}}
+
 apiVersion: networking.istio.io/v1beta1
 kind: DestinationRule
 metadata:
   name: {{ include "gateway-api-inference-extension.name" . }}
 spec:
-  host: {{ .Values.istio.destinationRule.host | default (printf "%s.%s.svc.cluster.local" (include "gateway-api-inference-extension.name" .) .Release.Namespace) }}
+  host: {{ (index $dr "host") | default (printf "%s.%s.svc.cluster.local" (include "gateway-api-inference-extension.name" .) .Release.Namespace) }}
   trafficPolicy:
     tls:
       mode: SIMPLE
       insecureSkipVerify: true
-    {{- if .Values.istio.destinationRule.trafficPolicy.connectionPool }}
-    connectionPool: 
-      {{- .Values.istio.destinationRule.trafficPolicy.connectionPool | toYaml | nindent 6 }}
+    {{- with (index (index $dr "trafficPolicy") "connectionPool") }}
+    connectionPool:
+      {{- toYaml . | nindent 6 }}
     {{- end }}
 {{- end }}
diff --git a/config/charts/inferencepool/values.yaml b/config/charts/inferencepool/values.yaml
index daf2bfc..bed1f46 100644
--- a/config/charts/inferencepool/values.yaml
+++ b/config/charts/inferencepool/values.yaml
@@ -12,6 +12,11 @@ inferenceExtension:
   extraContainerPorts: []
   # Define additional service ports
   extraServicePorts: []
+#  extraServicePorts:
+#    - name: http
+#      port: 8081
+#      protocol: TCP
+#      targetPort: 8081
 
   # This is the plugins configuration file.
   # pluginsCustomConfig:
@@ -38,6 +43,10 @@ inferenceExtension:
   affinity: {}
 
   tolerations: []
+  
+  # Sidecar configuration for EPP
+  sidecar:
+    enabled: false
 
   # Monitoring configuration for EPP
   monitoring:
@@ -85,12 +94,25 @@ provider:
     # Set to true if the cluster is an Autopilot cluster.
     autopilot: false
 
+  # Istio-specific configuration.
+  # This block is only used if name is "istio".
+  istio:
+    destinationRule:
+      # Provide a way to override the default calculated host
+      host: ""
+      # Optional: Enables customization of the traffic policy
+      trafficPolicy: {}
+        # connectionPool:
+        #   http:
+        #     maxRequestsPerConnection: 256000
+
+# DEPRECATED and will be removed in v1.3. Instead, use `provider.istio.*`.
 istio:
   destinationRule:
     # Provide a way to override the default calculated host
-    host: "" 
+    host: ""
     # Optional: Enables customization of the traffic policy
     trafficPolicy: {}
       # connectionPool:
       #   http:
-      #     maxRequestsPerConnection: 256000
\ No newline at end of file
+      #     maxRequestsPerConnection: 256000
diff --git a/config/crd/bases/inference.networking.x-k8s.io_inferencemodelrewrites.yaml b/config/crd/bases/inference.networking.x-k8s.io_inferencemodelrewrites.yaml
new file mode 100644
index 0000000..bb9b3e6
--- /dev/null
+++ b/config/crd/bases/inference.networking.x-k8s.io_inferencemodelrewrites.yaml
@@ -0,0 +1,237 @@
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  annotations:
+    inference.networking.k8s.io/bundle-version: main-dev
+  name: inferencemodelrewrites.inference.networking.x-k8s.io
+spec:
+  group: inference.networking.x-k8s.io
+  names:
+    kind: InferenceModelRewrite
+    listKind: InferenceModelRewriteList
+    plural: inferencemodelrewrites
+    singular: inferencemodelrewrite
+  scope: Namespaced
+  versions:
+  - additionalPrinterColumns:
+    - jsonPath: .spec.poolRef.name
+      name: Inference Pool
+      type: string
+    - jsonPath: .metadata.creationTimestamp
+      name: Age
+      type: date
+    name: v1alpha2
+    schema:
+      openAPIV3Schema:
+        description: InferenceModelRewrite is the Schema for the InferenceModelRewrite
+          API.
+        properties:
+          apiVersion:
+            description: |-
+              APIVersion defines the versioned schema of this representation of an object.
+              Servers should convert recognized schemas to the latest internal value, and
+              may reject unrecognized values.
+              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
+            type: string
+          kind:
+            description: |-
+              Kind is a string value representing the REST resource this object represents.
+              Servers may infer this from the endpoint the client submits requests to.
+              Cannot be updated.
+              In CamelCase.
+              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
+            type: string
+          metadata:
+            type: object
+          spec:
+            description: InferenceModelRewriteSpec defines the desired state of InferenceModelRewrite.
+            properties:
+              poolRef:
+                description: PoolRef is a reference to the inference pool.
+                properties:
+                  group:
+                    default: inference.networking.k8s.io
+                    description: Group is the group of the referent.
+                    maxLength: 253
+                    pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
+                    type: string
+                  kind:
+                    default: InferencePool
+                    description: Kind is kind of the referent. For example "InferencePool".
+                    maxLength: 63
+                    minLength: 1
+                    pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
+                    type: string
+                  name:
+                    description: Name is the name of the referent.
+                    maxLength: 253
+                    minLength: 1
+                    type: string
+                required:
+                - name
+                type: object
+              rules:
+                items:
+                  description: |-
+                    InferenceModelRewriteRule defines the match criteria and corresponding action.
+
+                    A specific model name can only be matched by one rule across all
+                    rules attached to the same InferencePool. If multiple rules attempt
+                    to match the same model name, the oldest rule (by creationTimestamp)
+                    will be the only one considered valid.
+                  properties:
+                    matches:
+                      items:
+                        description: Match defines the criteria for matching the LLM
+                          requests.
+                        properties:
+                          model:
+                            description: |-
+                              Model specifies the criteria for matching the 'model' field
+                              within the JSON request body.
+                            properties:
+                              type:
+                                default: Exact
+                                description: |-
+                                  Type specifies the kind of string matching to use.
+                                  Supported value is "Exact". Defaults to "Exact".
+                                enum:
+                                - Exact
+                                type: string
+                              value:
+                                description: Value is the model name string to match
+                                  against.
+                                minLength: 1
+                                type: string
+                            required:
+                            - value
+                            type: object
+                        required:
+                        - model
+                        type: object
+                      type: array
+                    split:
+                      items:
+                        description: TargetModel defines a weighted model destination
+                          for traffic distribution.
+                        properties:
+                          modelRewrite:
+                            type: string
+                          weight:
+                            description: |-
+                              (The following comment is copied from the original targetModel)
+                              Weight is used to determine the proportion of traffic that should be
+                              sent to this model when multiple target models are specified.
+
+                              Weight defines the proportion of requests forwarded to the specified
+                              model. This is computed as weight/(sum of all weights in this
+                              TargetModels list). For non-zero values, there may be some epsilon from
+                              the exact proportion defined here depending on the precision an
+                              implementation supports. Weight is not a percentage and the sum of
+                              weights does not need to equal 100.
+
+                              If a weight is set for any targetModel, it must be set for all targetModels.
+                              Conversely weights are optional, so long as ALL targetModels do not specify a weight.
+                            format: int32
+                            maximum: 1000000
+                            minimum: 1
+                            type: integer
+                        required:
+                        - modelRewrite
+                        type: object
+                      minItems: 1
+                      type: array
+                  type: object
+                type: array
+            required:
+            - poolRef
+            - rules
+            type: object
+          status:
+            description: InferenceModelRewriteStatus defines the observed state of
+              InferenceModelRewrite.
+            properties:
+              conditions:
+                default:
+                - lastTransitionTime: "1970-01-01T00:00:00Z"
+                  message: Waiting for controller
+                  reason: Pending
+                  status: Unknown
+                  type: Accepted
+                description: |-
+                  Conditions track the state of the InferenceModelRewrite.
+
+                  Known condition types are:
+
+                  * "Accepted"
+                items:
+                  description: Condition contains details for one aspect of the current
+                    state of this API Resource.
+                  properties:
+                    lastTransitionTime:
+                      description: |-
+                        lastTransitionTime is the last time the condition transitioned from one status to another.
+                        This should be when the underlying condition changed.  If that is not known, then using the time when the API field changed is acceptable.
+                      format: date-time
+                      type: string
+                    message:
+                      description: |-
+                        message is a human readable message indicating details about the transition.
+                        This may be an empty string.
+                      maxLength: 32768
+                      type: string
+                    observedGeneration:
+                      description: |-
+                        observedGeneration represents the .metadata.generation that the condition was set based upon.
+                        For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
+                        with respect to the current state of the instance.
+                      format: int64
+                      minimum: 0
+                      type: integer
+                    reason:
+                      description: |-
+                        reason contains a programmatic identifier indicating the reason for the condition's last transition.
+                        Producers of specific condition types may define expected values and meanings for this field,
+                        and whether the values are considered a guaranteed API.
+                        The value should be a CamelCase string.
+                        This field may not be empty.
+                      maxLength: 1024
+                      minLength: 1
+                      pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
+                      type: string
+                    status:
+                      description: status of the condition, one of True, False, Unknown.
+                      enum:
+                      - "True"
+                      - "False"
+                      - Unknown
+                      type: string
+                    type:
+                      description: type of condition in CamelCase or in foo.example.com/CamelCase.
+                      maxLength: 316
+                      pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
+                      type: string
+                  required:
+                  - lastTransitionTime
+                  - message
+                  - reason
+                  - status
+                  - type
+                  type: object
+                maxItems: 8
+                type: array
+                x-kubernetes-list-map-keys:
+                - type
+                x-kubernetes-list-type: map
+            type: object
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}
+status:
+  acceptedNames:
+    kind: ""
+    plural: ""
+  conditions: null
+  storedVersions: null
diff --git a/config/manifests/gateway/nginxgatewayfabric/gateway.yaml b/config/manifests/gateway/nginxgatewayfabric/gateway.yaml
new file mode 100644
index 0000000..cdc1308
--- /dev/null
+++ b/config/manifests/gateway/nginxgatewayfabric/gateway.yaml
@@ -0,0 +1,10 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: inference-gateway
+spec:
+  gatewayClassName: nginx
+  listeners:
+  - name: http
+    port: 80
+    protocol: HTTP
diff --git a/config/manifests/gateway/nginxgatewayfabric/httproute.yaml b/config/manifests/gateway/nginxgatewayfabric/httproute.yaml
new file mode 100644
index 0000000..70c3887
--- /dev/null
+++ b/config/manifests/gateway/nginxgatewayfabric/httproute.yaml
@@ -0,0 +1,18 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: llm-route
+  namespace: default
+spec:
+  parentRefs:
+  - name: inference-gateway
+  rules:
+  - matches:
+    - path:
+        type: PathPrefix
+        value: /
+    backendRefs:
+    - group: inference.networking.k8s.io
+      kind: InferencePool
+      name: vllm-llama3-8b-instruct
+
diff --git a/conformance/resources/base.yaml b/conformance/resources/base.yaml
index 2e1b378..3b4933c 100644
--- a/conformance/resources/base.yaml
+++ b/conformance/resources/base.yaml
@@ -77,7 +77,7 @@ spec:
     spec:
       containers:
       - name: echoserver
-        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
+        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20251106-v1.3.0-263-g47c3435c
         ports:
         - containerPort: 3000
         readinessProbe:
@@ -121,7 +121,7 @@ spec:
     spec:
       containers:
       - name: echoserver
-        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
+        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20251106-v1.3.0-263-g47c3435c
         ports:
         - containerPort: 3000
         readinessProbe:
@@ -200,7 +200,7 @@ spec:
       terminationGracePeriodSeconds: 130
       containers:
       - name: epp
-        image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:v1.0.0
+        image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:v20251105-cbb8928
         imagePullPolicy: Always
         args:
         - --pool-name
@@ -298,7 +298,7 @@ spec:
       terminationGracePeriodSeconds: 130
       containers:
       - name: epp
-        image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:v1.0.0
+        image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:v20251105-cbb8928
         imagePullPolicy: Always
         args:
         - --pool-name
@@ -340,6 +340,215 @@ spec:
         configMap:
           name: plugins-config
 ---
+# -- Data Parallelism (DP) backend deployment: 3 pods, each listening on three ports to simulate ranks ---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: dp-inference-model-server-deployment
+  namespace: inference-conformance-app-backend
+  labels:
+    app: dp-inference-model-server
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: dp-inference-model-server
+  template:
+    metadata:
+      labels:
+        app: dp-inference-model-server
+    spec:
+      containers:
+      - name: echoserver-3000
+        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20251106-v1.3.0-263-g47c3435c
+        ports:
+        - containerPort: 3000
+        readinessProbe:
+          httpGet:
+            path: /
+            port: 3000
+          initialDelaySeconds: 3
+          periodSeconds: 5
+          failureThreshold: 2
+        env:
+        - name: HTTP_PORT # Default port for HTTP echo server
+          value: "3000"
+        - name: H2C_PORT # Default port for HTC echo server
+          value: "3001"
+        - name: INCLUDE_HTTP_PORT_HEADER
+          value: "true"
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+        - name: POD_IP
+          valueFrom:
+            fieldRef:
+              fieldPath: status.podIP
+      - name: echoserver-3002
+        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20251106-v1.3.0-263-g47c3435c
+        ports:
+        - containerPort: 3002
+        readinessProbe:
+          httpGet:
+            path: /
+            port: 3002
+          initialDelaySeconds: 3
+          periodSeconds: 5
+          failureThreshold: 2
+        env:
+        - name: HTTP_PORT
+          value: "3002"
+        - name: H2C_PORT
+          value: "3003"
+        - name: INCLUDE_HTTP_PORT_HEADER
+          value: "true"
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+        - name: POD_IP
+          valueFrom:
+            fieldRef:
+              fieldPath: status.podIP
+      - name: echoserver-3004
+        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20251106-v1.3.0-263-g47c3435c
+        ports:
+        - containerPort: 3004
+        readinessProbe:
+          httpGet:
+            path: /
+            port: 3004
+          initialDelaySeconds: 3
+          periodSeconds: 5
+          failureThreshold: 2
+        env:
+        - name: HTTP_PORT
+          value: "3004"
+        - name: H2C_PORT
+          value: "3005"
+        - name: INCLUDE_HTTP_PORT_HEADER
+          value: "true"
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+        - name: POD_IP
+          valueFrom:
+            fieldRef:
+              fieldPath: status.podIP
+---
+# ---  Data Parallelism (DP) InferencePool Definition ---
+apiVersion: inference.networking.k8s.io/v1
+kind: InferencePool
+metadata:
+  name: dp-inference-pool
+  namespace: inference-conformance-app-backend
+spec:
+  selector:
+    matchLabels:
+      app: dp-inference-model-server
+  targetPorts:
+    - number: 3000
+    - number: 3002
+    - number: 3004
+  endpointPickerRef:
+    name: dp-endpoint-picker-svc
+    port:
+      number: 9002
+---
+# --- Data Parallelism (DP) Conformance EPP service Definition ---
+apiVersion: v1
+kind: Service
+metadata:
+  name: dp-endpoint-picker-svc
+  namespace: inference-conformance-app-backend
+spec:
+  selector:
+    app: dp-app-backend-epp
+  ports:
+    - protocol: TCP
+      port: 9002
+      targetPort: 9002
+      appProtocol: http2
+  type: ClusterIP
+---
+# --- Data Parallelism (DP) Conformance EPP Deployment ---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: dp-app-endpoint-picker
+  namespace: inference-conformance-app-backend
+  labels:
+    app: dp-app-backend-epp
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: dp-app-backend-epp
+  template:
+    metadata:
+      labels:
+        app: dp-app-backend-epp
+    spec:
+      # Conservatively, this timeout should mirror the longest grace period of the pods within the pool
+      terminationGracePeriodSeconds: 130
+      containers:
+      - name: epp
+        image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:v20251105-cbb8928
+        imagePullPolicy: Always
+        args:
+        - --pool-name
+        - "dp-inference-pool"
+        - --pool-namespace
+        - "inference-conformance-app-backend"
+        - --v
+        - "4"
+        - --zap-encoder
+        - "json"
+        - --grpc-port
+        - "9002"
+        - --grpc-health-port
+        - "9003"
+        - "--config-file"
+        - "/config/conformance-plugins.yaml"
+        ports:
+        - containerPort: 9002
+        - containerPort: 9003
+        - name: metrics
+          containerPort: 9090
+        livenessProbe:
+          grpc:
+            port: 9003
+            service: inference-extension
+          initialDelaySeconds: 5
+          periodSeconds: 10
+        readinessProbe:
+          grpc:
+            port: 9003
+            service: inference-extension
+          initialDelaySeconds: 5
+          periodSeconds: 10
+        volumeMounts:
+        - name: plugins-config-volume
+          mountPath: "/config"
+      volumes:
+      - name: plugins-config-volume
+        configMap:
+          name: plugins-config
+---
 apiVersion: v1
 kind: ConfigMap
 metadata:
diff --git a/docs/proposals/1816-inferenceomodelrewrite/README.md b/docs/proposals/1816-inferenceomodelrewrite/README.md
new file mode 100644
index 0000000..9f20b36
--- /dev/null
+++ b/docs/proposals/1816-inferenceomodelrewrite/README.md
@@ -0,0 +1,234 @@
+# Inference Pool Level Model Name Redirect and Traffic Splitting
+
+**Status:** Proposed
+
+For the full, detailed proposal, please see the [original proposal](https://docs.google.com/document/d/12yR_nAWM-Tg2ZmgGYX1h-dlUNi0AqYoACUjNElipl0M/edit?usp=sharing).
+
+## Summary
+
+The original `InferenceModel` API ([v1alpha2](https://github.com/capri-xiyue/gateway-api-inference-extension/blob/0189c333c2d4076f099fda81bc37f41996426fa9/apix/v1alpha2/inferencemodel_types.go)) provided initial support for model routing. This proposal reintroduces and enhances the core functionalities of model name redirection and traffic splitting within an inference pool. This is essential for modern use cases such as model name aliasing/versioning and granular traffic splitting for gradual rollouts.
+
+### Goals
+*   Enable configurable model redirection/aliasing within an `InferencePool`.
+*   Enable traffic splitting within an `InferencePool`.
+
+### Non-Goals
+*   Traffic splitting between `InferencePools` will be handled by `HTTPRoute`.
+
+## Proposal
+
+We propose introducing a new Custom Resource Definition (CRD), `InferenceModelRewrite`, to define rules for model redirection and traffic splitting. The Endpoint Picker Proxy (EPP) will be responsible for watching these resources and performing the necessary request body mutations and traffic distribution.
+
+This approach provides the cleanest separation of concerns:
+*   **BBR / `HTTPRoute`:** Handles "global" (inter-pool) routing. It directs the user-facing model name to the correct logical `InferencePool`.
+*   **EPP / `InferenceModelRewrite`:** Manages "local" (intra-pool) implementation details, such as routing to different model versions (`v1` vs. `v2`) within that pool.
+
+For a more detailed discussion on the execution architecture (BBR vs. EPP) and naming conventions, please refer to the [original proposal document](https://docs.google.com/document/d/12yR_nAWM-Tg2ZmgGYX1h-dlUNi0AqYoACUjNElipl0M/edit?usp=sharing).
+
+### CRD Specification
+
+```go
+// InferenceModelRewrite is the Schema for the InferenceModelRewrite API.
+// +kubebuilder:object:root=true
+// +kubebuilder:subresource:status
+// +kubebuilder:printcolumn:name="Inference Pool",type=string,JSONPath=`.spec.poolRef.name`
+// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
+// +genclient
+type InferenceModelRewrite struct {
+	metav1.TypeMeta   `json:",inline"`
+	metav1.ObjectMeta `json:"metadata,omitempty"`
+
+	Spec   InferenceModelRewriteSpec   `json:"spec,omitempty"`
+	Status InferenceModelRewriteStatus `json:"status,omitempty"`
+}
+
+// InferenceModelRewriteList contains a list of InferenceModelRewrite.
+//
+// +kubebuilder:object:root=true
+type InferenceModelRewriteList struct {
+	metav1.TypeMeta `json:",inline"`
+	metav1.ListMeta `json:"metadata,omitempty"`
+	Items           []InferenceModelRewrite `json:"items"`
+}
+
+// InferenceModelRewriteSpec defines the desired state of InferenceModelRewrite.
+type InferenceModelRewriteSpec struct {
+	// PoolRef is a reference to the inference pool.
+	// +kubebuilder:validation:Required
+	PoolRef *PoolObjectReference `json:"poolRef"`
+
+	// Rules are the ordered set of rules for rewriting inference requests.
+	// The first rule to match a request will be used.
+	//
+	// --- Precedence and Conflict Resolution ---
+	// If multiple InferenceModelRewrite resources target the same
+	// InferencePool, the controller will merge them based on precedence.
+	//
+	// **Timestamp Wins:** If two rules from different rewrite all matches,
+	// the rule from the *oldest*
+	// InferenceModelRewrite resource (determined by
+	// metadata.creationTimestamp) will be used.
+	// +required
+	Rules []InferenceModelRewriteRule `json:"rules"`
+}
+
+// InferenceModelRewriteRule defines the match criteria and corresponding action.
+//
+// A specific model name can only be matched by one rule across all
+// rewrites attached to the same InferencePool. If multiple rules attempt
+// to match the same model name, the oldest rule (by creationTimestamp)
+// will be the only one considered valid.
+type InferenceModelRewriteRule struct {
+	// Matches defines the criteria for matching a request.
+	// If multiple match criteria are specified, a request matches if
+	// ANY of the criteria are satisfied (logical OR).
+	//If empty, the rule matches all requests.
+
+	// +optional
+	Matches []Match `json:"matches,omitempty"`
+
+	// --- Actions ---
+	// Targets defines how to distribute traffic across a set of
+	// weighted model targets. This is used for traffic splitting, A/B tests,
+	// or canary rollouts.
+	// +optional
+	// +kubebuilder:validation:MinItems=1
+	//
+	Targets []TargetModel `json:"split,omitempty"`
+}
+
+// TargetModel defines a weighted model destination for traffic distribution.
+type TargetModel struct {
+	// (The following comment is copied from the original targetModel)
+	// Weight is used to determine the proportion of traffic that should be
+	// sent to this model when multiple target models are specified.
+	//
+	// Weight defines the proportion of requests forwarded to the specified
+	// model. This is computed as weight/(sum of all weights in this
+	// TargetModels list). For non-zero values, there may be some epsilon from
+	// the exact proportion defined here depending on the precision an
+	// implementation supports. Weight is not a percentage and the sum of
+	// weights does not need to equal 100.
+	//
+	// If a weight is set for any targetModel, it must be set for all targetModels.
+	// Conversely weights are optional, so long as ALL targetModels do not specify a weight.
+	//
+	// +optional
+	// +kubebuilder:validation:Minimum=1
+	// +kubebuilder:validation:Maximum=1000000
+	Weight int32 `json:"weight"`
+
+	// --- Destination Types ---
+	// ModelRewrite specifies a static model name destination.
+	// +required
+	ModelRewrite string `json:"modelRewrite"`
+}
+
+// Match defines the criteria for matching the LLM requests.
+type Match struct {
+	// Model specifies the criteria for matching the 'model' field
+	// within the JSON request body.
+	// +required
+	Model *ModelMatch `json:"model,omitempty"`
+}
+
+// ModelMatch defines how to match against the model name in the request body.
+type ModelMatch struct {
+	// Type specifies the kind of string matching to use.
+	// Supported value is "Exact". Defaults to "Exact".
+	// +optional
+	// +kubebuilder:default=Exact
+	Type *MatchValidationType `json:"type,omitempty"`
+
+	// Value is the model name string to match against.
+	// +required
+	// +kubebuilder:validation:MinLength=1
+	Value string `json:"value"`
+}
+
+// MatchValidationType specifies the type of string matching to use.
+// +kubebuilder:validation:Enum=Exact
+type MatchValidationType string
+
+const (
+	// MatchExact indicates that the model name must match exactly.
+	MatchExact MatchValidationType = "Exact"
+)
+
+// InferenceModelRewriteStatus defines the observed state of InferenceModelRewrite.
+type InferenceModelRewriteStatus struct {
+	// Conditions track the state of the InferenceModelRewrite.
+	//
+	// Known condition types are:
+	//
+	// * "Accepted"
+	//
+	// +optional
+	// +listType=map
+	// +listMapKey=type
+	// +kubebuilder:validation:MaxItems=8
+	// +kubebuilder:default={{type: "Accepted", status: "Unknown", reason:"Pending", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}
+	Conditions []metav1.Condition `json:"conditions,omitempty"`
+}
+
+// InferenceModelRewriteConditionType is a type of condition for the InferenceModelRewrite.
+type InferenceModelRewriteConditionType string
+
+// InferenceModelRewriteConditionReason is the reason for a given InferenceModelRewriteConditionType.
+type InferenceModelRewriteConditionReason string
+
+const (
+	// RewriteConditionAccepted indicates if the rewrite is accepted, and if not, why.
+	// This is the primary condition for this resource.
+	//
+	// Possible reasons for this condition to be True are:
+	//
+	// * "Accepted"
+	//
+	// Possible reasons for this condition to be Unknown are:
+	//
+	// * "Pending"
+	//
+	RewriteConditionAccepted InferenceModelRewriteConditionType = "Accepted"
+
+	// RewriteReasonAccepted indicates the rewrite is valid, non-conflicting,
+	// and has been successfully applied to the inference pool.
+	RewriteReasonAccepted InferenceModelRewriteConditionReason = "Accepted"
+
+	// RewriteReasonPending is the initial state, and indicates that the
+	// controller has not yet reconciled the InferenceModelRewrite.
+	RewriteReasonPending InferenceModelRewriteConditionReason = "Pending"
+)
+```
+
+### Example: Traffic Splitting
+
+```yaml
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: InferenceModelRewrite
+metadata:
+  name: food-review-canary-rollout
+spec:
+  poolRef:
+    name: main-food-review-pool
+  rules:
+  - matches:
+    - model:
+        type: Exact
+        value: "foodreview"
+    targets:
+    - modelRewrite: "foodreview-v1"
+      weight: 10
+    - modelRewrite: "foodreview-v2"
+      weight: 90
+```
+
+## Implementation Phases
+
+*   **Phase 1: EPP-Driven Intra-Pool Rewrite:** This phase delivers the core functionality for the most common use case: model rewrite and traffic splitting within a single `InferencePool`.
+    *   The `InferenceModelRewrite` CRD will be created.
+    *   The EPP will be enhanced to be a read-only controller for this CRD, executing the request body mutation and traffic splitting.
+    *   **Key Point:** The EPP will be a **read-only** consumer of the CRD and will not write to its status field. Status updates (e.g., marking conflicts) must be handled by a new, separate controller if needed.
+
+*   **Phase 2 (Conditional): Promote Rewrite Logic to a Shared Library:** If user patterns show a strong need for inter-pool traffic splitting based on rewritten model names, the logic can be moved into a shared library used by both BBR and EPP.
+    *   An idempotency mechanism, like an `X-Gateway-Model-Name` header, would be added to prevent the EPP from re-executing the logic.
diff --git a/go.mod b/go.mod
index b0641db..dc53db3 100644
--- a/go.mod
+++ b/go.mod
@@ -26,17 +26,17 @@ require (
 	golang.org/x/sync v0.18.0
 	google.golang.org/grpc v1.76.0
 	google.golang.org/protobuf v1.36.10
-	k8s.io/api v0.34.1
-	k8s.io/apiextensions-apiserver v0.34.1
-	k8s.io/apimachinery v0.34.1
-	k8s.io/client-go v0.34.1
-	k8s.io/code-generator v0.34.1
-	k8s.io/component-base v0.34.1
+	k8s.io/api v0.34.2
+	k8s.io/apiextensions-apiserver v0.34.2
+	k8s.io/apimachinery v0.34.2
+	k8s.io/client-go v0.34.2
+	k8s.io/code-generator v0.34.2
+	k8s.io/component-base v0.34.2
 	k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d
 	sigs.k8s.io/controller-runtime v0.22.4
 	// Update the CONTROLLER_TOOLS_VERSION in Makefile when bumping controller-tools.
 	sigs.k8s.io/controller-tools v0.19.0
-	sigs.k8s.io/gateway-api v1.4.0
+	sigs.k8s.io/gateway-api v1.3.1-0.20251106052652-079e4774d76b
 	sigs.k8s.io/structured-merge-diff/v6 v6.3.0
 	sigs.k8s.io/yaml v1.6.0
 )
@@ -109,14 +109,14 @@ require (
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
 	golang.org/x/crypto v0.43.0 // indirect
 	golang.org/x/exp v0.0.0-20250808145144-a408d31f581a // indirect
-	golang.org/x/mod v0.28.0 // indirect
+	golang.org/x/mod v0.29.0 // indirect
 	golang.org/x/net v0.46.0 // indirect
 	golang.org/x/oauth2 v0.32.0 // indirect
 	golang.org/x/sys v0.37.0 // indirect
 	golang.org/x/term v0.36.0 // indirect
 	golang.org/x/text v0.30.0 // indirect
 	golang.org/x/time v0.13.0 // indirect
-	golang.org/x/tools v0.37.0 // indirect
+	golang.org/x/tools v0.38.0 // indirect
 	gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20250922171735-9219d122eba9 // indirect
@@ -124,7 +124,7 @@ require (
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
-	k8s.io/apiserver v0.34.1 // indirect
+	k8s.io/apiserver v0.34.2 // indirect
 	k8s.io/gengo/v2 v2.0.0-20250820003526-c297c0c1eb9d // indirect
 	k8s.io/klog/v2 v2.130.1 // indirect
 	k8s.io/kube-openapi v0.0.0-20250814151709-d7b6acb124c3 // indirect
diff --git a/go.sum b/go.sum
index a425ef7..eea02a6 100644
--- a/go.sum
+++ b/go.sum
@@ -311,8 +311,8 @@ golang.org/x/exp v0.0.0-20250808145144-a408d31f581a h1:Y+7uR/b1Mw2iSXZ3G//1haIiS
 golang.org/x/exp v0.0.0-20250808145144-a408d31f581a/go.mod h1:rT6SFzZ7oxADUDx58pcaKFTcZ+inxAa9fTrYx/uVYwg=
 golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.28.0 h1:gQBtGhjxykdjY9YhZpSlZIsbnaE2+PgjfLWUQTnoZ1U=
-golang.org/x/mod v0.28.0/go.mod h1:yfB/L0NOf/kmEbXjzCPOx1iK1fRutOydrCMsqRhEBxI=
+golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=
+golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@@ -343,8 +343,8 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm
 golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE=
-golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w=
+golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
+golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
 golang.org/x/tools/go/expect v0.1.1-deprecated h1:jpBZDwmgPhXsKZC6WhL20P4b/wmnpsEAGHaNy0n/rJM=
 golang.org/x/tools/go/expect v0.1.1-deprecated/go.mod h1:eihoPOH+FgIqa3FpoTwguz/bVUSGBlGQU67vpBeOrBY=
 golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated h1:1h2MnaIAIXISqTFKdENegdpAgUXz6NrPEsbIeWaBRvM=
@@ -381,20 +381,20 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-k8s.io/api v0.34.1 h1:jC+153630BMdlFukegoEL8E/yT7aLyQkIVuwhmwDgJM=
-k8s.io/api v0.34.1/go.mod h1:SB80FxFtXn5/gwzCoN6QCtPD7Vbu5w2n1S0J5gFfTYk=
-k8s.io/apiextensions-apiserver v0.34.1 h1:NNPBva8FNAPt1iSVwIE0FsdrVriRXMsaWFMqJbII2CI=
-k8s.io/apiextensions-apiserver v0.34.1/go.mod h1:hP9Rld3zF5Ay2Of3BeEpLAToP+l4s5UlxiHfqRaRcMc=
-k8s.io/apimachinery v0.34.1 h1:dTlxFls/eikpJxmAC7MVE8oOeP1zryV7iRyIjB0gky4=
-k8s.io/apimachinery v0.34.1/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
-k8s.io/apiserver v0.34.1 h1:U3JBGdgANK3dfFcyknWde1G6X1F4bg7PXuvlqt8lITA=
-k8s.io/apiserver v0.34.1/go.mod h1:eOOc9nrVqlBI1AFCvVzsob0OxtPZUCPiUJL45JOTBG0=
-k8s.io/client-go v0.34.1 h1:ZUPJKgXsnKwVwmKKdPfw4tB58+7/Ik3CrjOEhsiZ7mY=
-k8s.io/client-go v0.34.1/go.mod h1:kA8v0FP+tk6sZA0yKLRG67LWjqufAoSHA2xVGKw9Of8=
-k8s.io/code-generator v0.34.1 h1:WpphT26E+j7tEgIUfFr5WfbJrktCGzB3JoJH9149xYc=
-k8s.io/code-generator v0.34.1/go.mod h1:DeWjekbDnJWRwpw3s0Jat87c+e0TgkxoR4ar608yqvg=
-k8s.io/component-base v0.34.1 h1:v7xFgG+ONhytZNFpIz5/kecwD+sUhVE6HU7qQUiRM4A=
-k8s.io/component-base v0.34.1/go.mod h1:mknCpLlTSKHzAQJJnnHVKqjxR7gBeHRv0rPXA7gdtQ0=
+k8s.io/api v0.34.2 h1:fsSUNZhV+bnL6Aqrp6O7lMTy6o5x2C4XLjnh//8SLYY=
+k8s.io/api v0.34.2/go.mod h1:MMBPaWlED2a8w4RSeanD76f7opUoypY8TFYkSM+3XHw=
+k8s.io/apiextensions-apiserver v0.34.2 h1:WStKftnGeoKP4AZRz/BaAAEJvYp4mlZGN0UCv+uvsqo=
+k8s.io/apiextensions-apiserver v0.34.2/go.mod h1:398CJrsgXF1wytdaanynDpJ67zG4Xq7yj91GrmYN2SE=
+k8s.io/apimachinery v0.34.2 h1:zQ12Uk3eMHPxrsbUJgNF8bTauTVR2WgqJsTmwTE/NW4=
+k8s.io/apimachinery v0.34.2/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
+k8s.io/apiserver v0.34.2 h1:2/yu8suwkmES7IzwlehAovo8dDE07cFRC7KMDb1+MAE=
+k8s.io/apiserver v0.34.2/go.mod h1:gqJQy2yDOB50R3JUReHSFr+cwJnL8G1dzTA0YLEqAPI=
+k8s.io/client-go v0.34.2 h1:Co6XiknN+uUZqiddlfAjT68184/37PS4QAzYvQvDR8M=
+k8s.io/client-go v0.34.2/go.mod h1:2VYDl1XXJsdcAxw7BenFslRQX28Dxz91U9MWKjX97fE=
+k8s.io/code-generator v0.34.2 h1:9bG6jTxmsU3HXE5BNYJTC8AZ1D6hVVfkm8yYSkdkGY0=
+k8s.io/code-generator v0.34.2/go.mod h1:dnDDEd6S/z4uZ+PG1aE58ySCi/lR4+qT3a4DddE4/2I=
+k8s.io/component-base v0.34.2 h1:HQRqK9x2sSAsd8+R4xxRirlTjowsg6fWCPwWYeSvogQ=
+k8s.io/component-base v0.34.2/go.mod h1:9xw2FHJavUHBFpiGkZoKuYZ5pdtLKe97DEByaA+hHbM=
 k8s.io/gengo/v2 v2.0.0-20250820003526-c297c0c1eb9d h1:qUrYOinhdAUL0xxhA4gPqogPBaS9nIq2l2kTb6pmeB0=
 k8s.io/gengo/v2 v2.0.0-20250820003526-c297c0c1eb9d/go.mod h1:EJykeLsmFC60UQbYJezXkEsG2FLrt0GPNkU5iK5GWxU=
 k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
@@ -409,8 +409,8 @@ sigs.k8s.io/controller-runtime v0.22.4 h1:GEjV7KV3TY8e+tJ2LCTxUTanW4z/FmNB7l327U
 sigs.k8s.io/controller-runtime v0.22.4/go.mod h1:+QX1XUpTXN4mLoblf4tqr5CQcyHPAki2HLXqQMY6vh8=
 sigs.k8s.io/controller-tools v0.19.0 h1:OU7jrPPiZusryu6YK0jYSjPqg8Vhf8cAzluP9XGI5uk=
 sigs.k8s.io/controller-tools v0.19.0/go.mod h1:y5HY/iNDFkmFla2CfQoVb2AQXMsBk4ad84iR1PLANB0=
-sigs.k8s.io/gateway-api v1.4.0 h1:ZwlNM6zOHq0h3WUX2gfByPs2yAEsy/EenYJB78jpQfQ=
-sigs.k8s.io/gateway-api v1.4.0/go.mod h1:AR5RSqciWP98OPckEjOjh2XJhAe2Na4LHyXD2FUY7Qk=
+sigs.k8s.io/gateway-api v1.3.1-0.20251106052652-079e4774d76b h1:CoVExRHGK0xoewqKEtip+CXa/PPIxiEKeNJaKdEz5o0=
+sigs.k8s.io/gateway-api v1.3.1-0.20251106052652-079e4774d76b/go.mod h1:eEYVpDGr0WPqR/35ZTBIWWpwKL7uUzOqlT92mmv3fus=
 sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 h1:IpInykpT6ceI+QxKBbEflcR5EXP7sU1kvOlxwZh5txg=
 sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
 sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU=
diff --git a/latencypredictor/build-deploy.sh b/latencypredictor/build-deploy.sh
old mode 100755
new mode 100644
index 27aaa56..31041d6
--- a/latencypredictor/build-deploy.sh
+++ b/latencypredictor/build-deploy.sh
@@ -1,5 +1,3 @@
-#!/bin/bash
-
 # Copyright 2025 The Kubernetes Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -13,6 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
 set -e
 
 # Configuration
@@ -21,6 +20,7 @@ REGION="your-gcp-region"
 REPOSITORY="your-artifact-registry-repo"
 TRAINING_IMAGE="latencypredictor-training-server"
 PREDICTION_IMAGE="latencypredictor-prediction-server"
+TEST_IMAGE="latencypredictor-test"
 TAG="latest"
 
 # Colors for output
@@ -53,7 +53,18 @@ check_files() {
         fi
     done
     
-    echo_status "All required files found."
+    # Check for test-specific files
+    local test_files=("Dockerfile-test")
+    for file in "${test_files[@]}"; do
+        if [[ ! -f "$file" ]]; then
+            echo_warning "Test file $file not found - test image will not be built"
+            TEST_BUILD_ENABLED=false
+            return
+        fi
+    done
+    
+    TEST_BUILD_ENABLED=true
+    echo_status "All required files found (including test files)."
 }
 
 # Build Docker images
@@ -62,7 +73,7 @@ build_images() {
     
     # Build training server image
     echo_status "Building training server image..."
-    docker build -f Dockerfile-training  -t ${TRAINING_IMAGE}:${TAG} .
+    docker build -f Dockerfile-training -t ${TRAINING_IMAGE}:${TAG} .
 
     # Tag for training server
     docker tag ${TRAINING_IMAGE}:${TAG} \
@@ -76,7 +87,19 @@ build_images() {
     docker tag ${PREDICTION_IMAGE}:${TAG} \
         us-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/${PREDICTION_IMAGE}:${TAG}
     
-    echo_status "Images built successfully."
+    # Build test image if enabled
+    if [[ "$TEST_BUILD_ENABLED" == "true" ]]; then
+        echo_status "Building test image..."
+        docker build -f Dockerfile-test -t ${TEST_IMAGE}:${TAG} .
+
+        # Tag for test image
+        docker tag ${TEST_IMAGE}:${TAG} \
+            us-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/${TEST_IMAGE}:${TAG}
+        
+        echo_status "All images (including test) built successfully."
+    else
+        echo_status "Images built successfully (test image skipped)."
+    fi
 }
 
 # Push images to Artifact Registry
@@ -94,7 +117,14 @@ push_images() {
     echo_status "Pushing prediction server image..."
     docker push us-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/${PREDICTION_IMAGE}:${TAG}
     
-    echo_status "Images pushed successfully."
+    # Push test image if enabled
+    if [[ "$TEST_BUILD_ENABLED" == "true" ]]; then
+        echo_status "Pushing test image..."
+        docker push us-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/${TEST_IMAGE}:${TAG}
+        echo_status "All images (including test) pushed successfully."
+    else
+        echo_status "Images pushed successfully (test image skipped)."
+    fi
 }
 
 # Deploy to GKE
@@ -114,6 +144,112 @@ deploy_to_gke() {
     echo_status "Deployment completed successfully."
 }
 
+# Deploy test job
+deploy_test() {
+    echo_status "Deploying test job..."
+    
+    if [[ "$TEST_BUILD_ENABLED" != "true" ]]; then
+        echo_warning "Test image not available. Skipping test deployment."
+        return
+    fi
+    
+    # Check if test manifest exists
+    if [[ ! -f "test-job.yaml" ]]; then
+        echo_warning "test-job.yaml not found. Creating a basic test job..."
+        create_test_manifest
+    fi
+    
+    # Delete existing test job if it exists
+    kubectl delete job latency-predictor-test --ignore-not-found=true
+    
+    # Apply test job
+    kubectl apply -f test-job.yaml
+    
+    echo_status "Test job deployed. Monitor with: kubectl logs -f job/latency-predictor-test"
+}
+
+# Create a basic test manifest
+create_test_manifest() {
+    cat > test-job.yaml << EOF
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: latency-predictor-test
+  namespace: default
+  labels:
+    app: latency-predictor-test
+    component: test
+spec:
+  template:
+    metadata:
+      labels:
+        app: latency-predictor-test
+        component: test
+    spec:
+      nodeSelector:
+        cloud.google.com/gke-nodepool: "pool-2"
+      restartPolicy: Never
+      containers:
+      - name: test-runner
+        image: us-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/${TEST_IMAGE}:${TAG}
+        imagePullPolicy: Always
+        command: ["pytest"]
+        args: ["-v", "-s", "test_dual_server_client.py"]
+        resources:
+          requests:
+            cpu: "500m"
+            memory: "1Gi"
+          limits:
+            cpu: "1000m"
+            memory: "2Gi"
+        env:
+        - name: TRAINING_SERVER_URL
+          value: "http://training-service:8000"
+        - name: PREDICTION_SERVER_URL
+          value: "http://prediction-service:80"
+        - name: TEST_TIMEOUT
+          value: "300"
+        volumeMounts:
+        - name: test-results
+          mountPath: /test-results
+      volumes:
+      - name: test-results
+        emptyDir: {}
+  backoffLimit: 3
+EOF
+    echo_status "Created basic test-job.yaml manifest."
+}
+
+# Run tests
+run_tests() {
+    echo_status "Running tests..."
+    
+    if [[ "$TEST_BUILD_ENABLED" != "true" ]]; then
+        echo_warning "Test image not available. Running basic connectivity tests instead..."
+        test_deployment
+        return
+    fi
+    
+    # Deploy and run test job
+    deploy_test
+    
+    # Wait for job completion and show logs
+    echo_status "Waiting for test job to complete..."
+    kubectl wait --for=condition=complete job/latency-predictor-test --timeout=600s || {
+        echo_error "Test job did not complete successfully"
+        kubectl describe job latency-predictor-test
+        kubectl logs job/latency-predictor-test
+        return 1
+    }
+    
+    echo_status "Test job completed. Showing logs:"
+    kubectl logs job/latency-predictor-test
+    
+    # Clean up test job
+    echo_status "Cleaning up test job..."
+    kubectl delete job latency-predictor-test
+}
+
 # Get service information
 get_service_info() {
     echo_status "Getting service information..."
@@ -143,7 +279,7 @@ get_service_info() {
     kubectl get services
 }
 
-# Test the deployment
+# Test the deployment (basic connectivity tests)
 test_deployment() {
     echo_status "Testing deployment..."
     
@@ -177,6 +313,18 @@ test_deployment() {
     fi
 }
 
+# List built images
+list_images() {
+    echo_status "Listing built images..."
+    
+    echo_status "Local images:"
+    docker images | grep -E "${TRAINING_IMAGE}|${PREDICTION_IMAGE}|${TEST_IMAGE}" || echo "No local images found"
+    
+    echo_status "Remote images in Artifact Registry:"
+    gcloud artifacts docker images list us-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY} \
+        --include-tags --filter="package~(${TRAINING_IMAGE}|${PREDICTION_IMAGE}|${TEST_IMAGE})" || echo "No remote images found"
+}
+
 # Cleanup function
 cleanup() {
     echo_status "Cleaning up..."
@@ -196,15 +344,27 @@ main() {
             build_images
             ;;
         "push")
+            check_files
             push_images
             ;;
         "deploy")
             deploy_to_gke
             ;;
+        "test-deploy")
+            check_files
+            deploy_test
+            ;;
+        "test")
+            check_files
+            run_tests
+            ;;
         "info")
             get_service_info
             ;;
-        "test")
+        "images")
+            list_images
+            ;;
+        "basic-test")
             test_deployment
             ;;
         "all")
@@ -216,17 +376,30 @@ main() {
             test_deployment
             cleanup
             ;;
+        "full")
+            check_files
+            build_images
+            push_images
+            deploy_to_gke
+            get_service_info
+            run_tests
+            cleanup
+            ;;
         *)
-            echo "Usage: $0 {check|build|push|deploy|info|test|all}"
+            echo "Usage: $0 {check|build|push|deploy|test-deploy|test|info|images|basic-test|all|full}"
             echo ""
             echo "Commands:"
-            echo "  check  - Check if required files exist"
-            echo "  build  - Build Docker images"
-            echo "  push   - Push images to Artifact Registry"
-            echo "  deploy - Deploy to GKE"
-            echo "  info   - Get service information"
-            echo "  test   - Test the deployment"
-            echo "  all    - Run complete build and deployment process"
+            echo "  check      - Check if required files exist"
+            echo "  build      - Build Docker images (including test if Dockerfile-test exists)"
+            echo "  push       - Push images to Artifact Registry"
+            echo "  deploy     - Deploy to GKE"
+            echo "  test-deploy- Deploy test job only"
+            echo "  test       - Run comprehensive tests using test image"
+            echo "  info       - Get service information"
+            echo "  images     - List built images (local and remote)"
+            echo "  basic-test - Run basic connectivity tests"
+            echo "  all        - Run complete build and deployment process (no tests)"
+            echo "  full       - Run complete process including comprehensive tests"
             exit 1
             ;;
     esac
diff --git a/latencypredictor/training_server.py b/latencypredictor/training_server.py
index 94a656c..3e1e275 100644
--- a/latencypredictor/training_server.py
+++ b/latencypredictor/training_server.py
@@ -552,20 +552,43 @@ class LatencyPredictor:
             logging.error(f"Error in _train_model_with_scaling: {e}", exc_info=True)
             raise
         
-    def _calculate_quantile_metrics_on_test(self, model, scaler, test_data, feature_cols, target_col):
+    def _calculate_quantile_metrics_on_test(self, model, scaler, test_data, model_name, target_col):
         """Calculate quantile-specific metrics on test data"""
         try:
-            df = pd.DataFrame(test_data).dropna()
-            df = df[df[target_col] > 0]
-            
-            if len(df) < 2:
+            df_raw = pd.DataFrame(test_data).dropna()
+            df_raw = df_raw[df_raw[target_col] > 0]
+
+            if len(df_raw) < 2:
                 return None, None, None
+
+        # Apply feature engineering to create interaction terms and categorical features
+            df_features = self._prepare_features_with_interaction(df_raw.copy(), model_type=model_name)
+
+        # Get appropriate feature columns based on model type and name
+            if model_name == "ttft":
+                if self.model_type == ModelType.BAYESIAN_RIDGE:
+                    feature_cols = [
+                        'kv_cache_percentage','input_token_length','num_request_waiting',
+                        'num_request_running','prefix_cache_score','effective_input_tokens'
+                    ]
+                else:  # XGBoost or LightGBM
+                    feature_cols = [
+                        'kv_cache_percentage','input_token_length','num_request_waiting',
+                        'num_request_running','prefix_cache_score','effective_input_tokens','prefill_score_bucket'
+                    ]
+            else:  # tpot
+                feature_cols = ['kv_cache_percentage', 'input_token_length', 
+                   'num_request_waiting', 'num_request_running', 'num_tokens_generated']
+
+            X = df_features[feature_cols]  # ✅ Now has properly typed categorical!
             
-            X = df[feature_cols]
+
+            
+
             if self.model_type == ModelType.BAYESIAN_RIDGE and scaler is not None:
                 X = scaler.transform(X)
-            
-            y_true = df[target_col].values
+
+            y_true = df_raw[target_col].values
             y_pred = model.predict(X)
             
             # For Bayesian Ridge (which doesn't do true quantile regression), 
@@ -644,19 +667,19 @@ class LatencyPredictor:
                     'kv_cache_percentage','input_token_length','num_request_waiting',
                     'num_request_running','prefix_cache_score','effective_input_tokens','prefill_score_bucket'
                 ]
-                ttft_feature_cols_br = [
+                    ttft_feature_cols_br = [
                     'kv_cache_percentage','input_token_length','num_request_waiting',
                     'num_request_running','prefix_cache_score','effective_input_tokens'
                 ]
 
-                # Build X_ttft for all model types, then trim for BR
-                X_ttft = df_ttft[ttft_feature_cols_tree]
-                if self.model_type == ModelType.BAYESIAN_RIDGE:
-                    X_ttft = X_ttft[ttft_feature_cols_br]
+                    # Build X_ttft for all model types, then trim for BR
+                    X_ttft = df_ttft[ttft_feature_cols_tree]
+                    if self.model_type == ModelType.BAYESIAN_RIDGE:
+                        X_ttft = X_ttft[ttft_feature_cols_br]
 
-                y_ttft = raw_ttft['actual_ttft_ms']
+                    y_ttft = raw_ttft['actual_ttft_ms']
 
-                try:
+                    try:
                         # raw_ttft still has the original columns including 'prefix_cache_score'
                         raw_ttft['_prefix_bucket'] = raw_ttft['prefix_cache_score'].clip(0, 1).apply(
                             lambda s: min(int(s * self.prefix_buckets), self.prefix_buckets - 1)
@@ -681,14 +704,13 @@ class LatencyPredictor:
 
                         # Quantile metrics on test set
                         ql = coverage = violation_rate = None
-                        ttft_test_raw = pd.DataFrame(list(self.ttft_test_data)).dropna()
-                        if not ttft_test_raw.empty:
-                            ttft_test_fe = self._prepare_features_with_interaction(ttft_test_raw.copy(), "ttft")
-                            cols = ttft_feature_cols_br if self.model_type == ModelType.BAYESIAN_RIDGE else ttft_feature_cols_tree
-                            test_records = ttft_test_fe[cols].assign(actual_ttft_ms=ttft_test_raw['actual_ttft_ms']).to_dict("records")
+                        if self.ttft_test_data:
                             ql, coverage, violation_rate = self._calculate_quantile_metrics_on_test(
-                                new_ttft_model, new_ttft_scaler, test_records, cols, 'actual_ttft_ms'
-                            )
+                                new_ttft_model, new_ttft_scaler,
+                                list(self.ttft_test_data),  # Pass raw data
+                                "ttft",                      # Pass model name instead of feature_cols
+                                'actual_ttft_ms'
+                            )       
 
 
                         
@@ -703,7 +725,7 @@ class LatencyPredictor:
                         else:
                             logging.info(f"TTFT model trained on {len(df_ttft)} samples. Quantile metrics = N/A (insufficient test data)")
 
-                except Exception:
+                    except Exception:
                         logging.error("Error training TTFT model", exc_info=True)
 
 
@@ -724,11 +746,12 @@ class LatencyPredictor:
                             new_tpot_scaler = None
                         
                         # Calculate quantile metrics on test data
-                        tpot_feature_cols = ['kv_cache_percentage', 'input_token_length', 'num_request_waiting', 'num_request_running', 'num_tokens_generated']
                         ql, coverage, violation_rate = self._calculate_quantile_metrics_on_test(
                             new_tpot_model, new_tpot_scaler, 
-                            list(self.tpot_test_data), tpot_feature_cols, 'actual_tpot_ms'
-                        )
+                            list(self.tpot_test_data),  # Pass raw data
+                        "tpot",                      # Pass model name instead of feature_cols
+                        'actual_tpot_ms'
+                )
                         
                         if ql is not None:
                             self.tpot_quantile_loss_scores.append(ql)
diff --git a/mkdocs.yml b/mkdocs.yml
index 83b3a1a..9fca3cb 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -73,7 +73,7 @@ nav:
         - InferencePool Rollout: guides/inferencepool-rollout.md
       - Metrics and Observability: guides/metrics-and-observability.md
       - Configuration Guide:
-          - Configuring the plugins via configuration YAML file: guides/epp-configuration/config-text.md
+          - Configuring the EndPoint Picker via configuration YAML file: guides/epp-configuration/config-text.md
           - Prefix Cache Aware Plugin: guides/epp-configuration/prefix-aware.md
       - Migration Guide: guides/ga-migration.md
       - Troubleshooting Guide: guides/troubleshooting.md
diff --git a/pkg/epp/backend/metrics/fake.go b/pkg/epp/backend/metrics/fake.go
index 613ebf5..1c7a905 100644
--- a/pkg/epp/backend/metrics/fake.go
+++ b/pkg/epp/backend/metrics/fake.go
@@ -32,8 +32,9 @@ import (
 
 // FakePodMetrics is an implementation of PodMetrics that doesn't run the async refresh loop.
 type FakePodMetrics struct {
-	Pod     *backend.Pod
-	Metrics *MetricsState
+	Pod        *backend.Pod
+	Metrics    *MetricsState
+	Attributes *datalayer.Attributes
 }
 
 func (fpm *FakePodMetrics) String() string {
@@ -51,6 +52,9 @@ func (fpm *FakePodMetrics) GetMetrics() *MetricsState {
 func (fpm *FakePodMetrics) UpdatePod(pod *datalayer.PodInfo) {
 	fpm.Pod = pod
 }
+func (fpm *FakePodMetrics) GetAttributes() *datalayer.Attributes {
+	return fpm.Attributes
+}
 
 func (*FakePodMetrics) Put(string, datalayer.Cloneable)        {}
 func (*FakePodMetrics) Get(string) (datalayer.Cloneable, bool) { return nil, false }
diff --git a/pkg/epp/backend/metrics/pod_metrics.go b/pkg/epp/backend/metrics/pod_metrics.go
index a1114ae..4d22ef1 100644
--- a/pkg/epp/backend/metrics/pod_metrics.go
+++ b/pkg/epp/backend/metrics/pod_metrics.go
@@ -126,6 +126,9 @@ func (pm *podMetrics) stopRefreshLoop() {
 func (*podMetrics) Put(string, datalayer.Cloneable)        {}
 func (*podMetrics) Get(string) (datalayer.Cloneable, bool) { return nil, false }
 func (*podMetrics) Keys() []string                         { return nil }
+func (*podMetrics) GetAttributes() *datalayer.Attributes {
+	return nil
+}
 
 func (pm *podMetrics) UpdateMetrics(updated *MetricsState) {
 	updated.UpdateTime = time.Now()
diff --git a/pkg/epp/config/config.go b/pkg/epp/config/config.go
index 7b07a07..59a5962 100644
--- a/pkg/epp/config/config.go
+++ b/pkg/epp/config/config.go
@@ -16,9 +16,15 @@ limitations under the License.
 
 package config
 
-import "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
+import (
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
+)
 
 // Config is the configuration loaded from the text based configuration
 type Config struct {
-	SchedulerConfig *scheduling.SchedulerConfig
+	SchedulerConfig          *scheduling.SchedulerConfig
+	SaturationDetectorConfig *saturationdetector.Config
 }
+
+type FeatureConfig map[string]bool
diff --git a/pkg/epp/config/loader/configloader.go b/pkg/epp/config/loader/configloader.go
index 865eae2..3488b25 100644
--- a/pkg/epp/config/loader/configloader.go
+++ b/pkg/epp/config/loader/configloader.go
@@ -29,6 +29,7 @@ import (
 	configapi "sigs.k8s.io/gateway-api-inference-extension/apix/config/v1alpha1"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
@@ -36,21 +37,35 @@ import (
 
 var scheme = runtime.NewScheme()
 
+var registeredFeatureGates = map[string]struct{}{}
+
 func init() {
 	utilruntime.Must(configapi.Install(scheme))
 }
 
-// Load config from supplied text that was converted to []byte
-func LoadConfig(configBytes []byte, handle plugins.Handle, logger logr.Logger) (*config.Config, error) {
+// LoadConfigPhaseOne first phase of loading configuration from supplied text that was converted to []byte
+func LoadConfigPhaseOne(configBytes []byte, logger logr.Logger) (*configapi.EndpointPickerConfig, config.FeatureConfig, error) {
 	rawConfig, err := loadRawConfig(configBytes)
 	if err != nil {
-		return nil, err
+		return nil, nil, err
 	}
 
 	logger.Info("Loaded configuration", "config", rawConfig)
 
+	if err = validateFeatureGates(rawConfig.FeatureGates); err != nil {
+		return nil, nil, fmt.Errorf("failed to validate feature gates - %w", err)
+	}
+
 	setDefaultsPhaseOne(rawConfig)
 
+	featureConfig := loadFeatureConfig(rawConfig.FeatureGates)
+
+	return rawConfig, featureConfig, nil
+}
+
+// LoadConfigPhaseOne first phase of loading configuration from supplied text that was converted to []byte
+func LoadConfigPhaseTwo(rawConfig *configapi.EndpointPickerConfig, handle plugins.Handle, logger logr.Logger) (*config.Config, error) {
+	var err error
 	// instantiate loaded plugins
 	if err = instantiatePlugins(rawConfig.Plugins, handle); err != nil {
 		return nil, fmt.Errorf("failed to instantiate plugins - %w", err)
@@ -70,6 +85,7 @@ func LoadConfig(configBytes []byte, handle plugins.Handle, logger logr.Logger) (
 	if err != nil {
 		return nil, err
 	}
+	config.SaturationDetectorConfig = loadSaturationDetectorConfig(rawConfig.SaturationDetector)
 
 	return config, nil
 }
@@ -121,6 +137,39 @@ func loadSchedulerConfig(configProfiles []configapi.SchedulingProfile, handle pl
 	return scheduling.NewSchedulerConfig(profileHandler, profiles), nil
 }
 
+func loadFeatureConfig(featureGates configapi.FeatureGates) map[string]bool {
+	featureConfig := map[string]bool{}
+
+	for gate := range registeredFeatureGates {
+		featureConfig[gate] = false
+	}
+
+	for _, gate := range featureGates {
+		featureConfig[gate] = true
+	}
+
+	return featureConfig
+}
+
+func loadSaturationDetectorConfig(sd *configapi.SaturationDetector) *saturationdetector.Config {
+	sdConfig := saturationdetector.Config{}
+
+	sdConfig.QueueDepthThreshold = sd.QueueDepthThreshold
+	if sdConfig.QueueDepthThreshold <= 0 {
+		sdConfig.QueueDepthThreshold = saturationdetector.DefaultQueueDepthThreshold
+	}
+	sdConfig.KVCacheUtilThreshold = sd.KVCacheUtilThreshold
+	if sdConfig.KVCacheUtilThreshold <= 0.0 || sdConfig.KVCacheUtilThreshold >= 1.0 {
+		sdConfig.KVCacheUtilThreshold = saturationdetector.DefaultKVCacheUtilThreshold
+	}
+	sdConfig.MetricsStalenessThreshold = sd.MetricsStalenessThreshold.Duration
+	if sdConfig.MetricsStalenessThreshold <= 0.0 {
+		sdConfig.MetricsStalenessThreshold = saturationdetector.DefaultMetricsStalenessThreshold
+	}
+
+	return &sdConfig
+}
+
 func instantiatePlugins(configuredPlugins []configapi.PluginSpec, handle plugins.Handle) error {
 	pluginNames := sets.New[string]() // set of plugin names, a name must be unique
 
@@ -150,34 +199,7 @@ func instantiatePlugins(configuredPlugins []configapi.PluginSpec, handle plugins
 	return nil
 }
 
-func validateSchedulingProfiles(config *configapi.EndpointPickerConfig) error {
-	profileNames := sets.New[string]()
-	for _, profile := range config.SchedulingProfiles {
-		if profile.Name == "" {
-			return errors.New("SchedulingProfile must have a name")
-		}
-
-		if profileNames.Has(profile.Name) {
-			return fmt.Errorf("the name '%s' has been specified for more than one SchedulingProfile", profile.Name)
-		}
-		profileNames.Insert(profile.Name)
-
-		for _, plugin := range profile.Plugins {
-			if len(plugin.PluginRef) == 0 {
-				return fmt.Errorf("SchedulingProfile '%s' plugins must have a plugin reference", profile.Name)
-			}
-
-			notFound := true
-			for _, pluginConfig := range config.Plugins {
-				if plugin.PluginRef == pluginConfig.Name {
-					notFound = false
-					break
-				}
-			}
-			if notFound {
-				return errors.New(plugin.PluginRef + " is a reference to an undefined Plugin")
-			}
-		}
-	}
-	return nil
+// RegisterFeatureGate registers feature gate keys for validation
+func RegisterFeatureGate(gate string) {
+	registeredFeatureGates[gate] = struct{}{}
 }
diff --git a/pkg/epp/config/loader/defaults.go b/pkg/epp/config/loader/defaults.go
index 2a8236e..17075dd 100644
--- a/pkg/epp/config/loader/defaults.go
+++ b/pkg/epp/config/loader/defaults.go
@@ -17,8 +17,10 @@ limitations under the License.
 package loader
 
 import (
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	configapi "sigs.k8s.io/gateway-api-inference-extension/apix/config/v1alpha1"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
@@ -41,6 +43,8 @@ var defaultScorerWeight = DefaultScorerWeight
 // setDefaultsPhaseOne Performs the first phase of setting configuration defaults.
 // In particuylar it:
 //  1. Sets the name of plugins, for which one wasn't specified
+//  2. Sets defaults for the feature gates
+//  3. Sets defaults for the SaturationDetector configuration
 func setDefaultsPhaseOne(cfg *configapi.EndpointPickerConfig) {
 	// If no name was given for the plugin, use it's type as the name
 	for idx, pluginConfig := range cfg.Plugins {
@@ -48,6 +52,26 @@ func setDefaultsPhaseOne(cfg *configapi.EndpointPickerConfig) {
 			cfg.Plugins[idx].Name = pluginConfig.Type
 		}
 	}
+
+	// If no feature gates were specified, provide a default FeatureGates struct
+	if cfg.FeatureGates == nil {
+		cfg.FeatureGates = configapi.FeatureGates{}
+	}
+
+	// If the SaturationDetector configuration wasn't specified setup a default one
+	if cfg.SaturationDetector == nil {
+		cfg.SaturationDetector = &configapi.SaturationDetector{}
+	}
+	if cfg.SaturationDetector.QueueDepthThreshold == 0 {
+		cfg.SaturationDetector.QueueDepthThreshold = saturationdetector.DefaultQueueDepthThreshold
+	}
+	if cfg.SaturationDetector.KVCacheUtilThreshold == 0.0 {
+		cfg.SaturationDetector.KVCacheUtilThreshold = saturationdetector.DefaultKVCacheUtilThreshold
+	}
+	if cfg.SaturationDetector.MetricsStalenessThreshold.Duration == 0.0 {
+		cfg.SaturationDetector.MetricsStalenessThreshold =
+			metav1.Duration{Duration: saturationdetector.DefaultMetricsStalenessThreshold}
+	}
 }
 
 // setDefaultsPhaseTwo Performs the second phase of setting configuration defaults.
diff --git a/pkg/epp/config/loader/validation.go b/pkg/epp/config/loader/validation.go
new file mode 100644
index 0000000..e43fcb5
--- /dev/null
+++ b/pkg/epp/config/loader/validation.go
@@ -0,0 +1,71 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package loader
+
+import (
+	"errors"
+	"fmt"
+
+	"k8s.io/apimachinery/pkg/util/sets"
+	configapi "sigs.k8s.io/gateway-api-inference-extension/apix/config/v1alpha1"
+)
+
+func validateSchedulingProfiles(config *configapi.EndpointPickerConfig) error {
+	profileNames := sets.New[string]()
+	for _, profile := range config.SchedulingProfiles {
+		if profile.Name == "" {
+			return errors.New("SchedulingProfile must have a name")
+		}
+
+		if profileNames.Has(profile.Name) {
+			return fmt.Errorf("the name '%s' has been specified for more than one SchedulingProfile", profile.Name)
+		}
+		profileNames.Insert(profile.Name)
+
+		for _, plugin := range profile.Plugins {
+			if len(plugin.PluginRef) == 0 {
+				return fmt.Errorf("SchedulingProfile '%s' plugins must have a plugin reference", profile.Name)
+			}
+
+			notFound := true
+			for _, pluginConfig := range config.Plugins {
+				if plugin.PluginRef == pluginConfig.Name {
+					notFound = false
+					break
+				}
+			}
+			if notFound {
+				return errors.New(plugin.PluginRef + " is a reference to an undefined Plugin")
+			}
+		}
+	}
+	return nil
+}
+
+func validateFeatureGates(fg configapi.FeatureGates) error {
+	if fg == nil {
+		return nil
+	}
+
+	for _, gate := range fg {
+		if _, ok := registeredFeatureGates[gate]; !ok {
+			return errors.New(gate + " is an unregistered Feature Gate")
+		}
+	}
+
+	return nil
+}
diff --git a/pkg/epp/datalayer/datasource.go b/pkg/epp/datalayer/datasource.go
index 737e2c5..40fd365 100644
--- a/pkg/epp/datalayer/datasource.go
+++ b/pkg/epp/datalayer/datasource.go
@@ -26,7 +26,10 @@ import (
 
 // DataSource provides raw data to registered Extractors.
 type DataSource interface {
+	// Name of this data source.
 	Name() string
+	// Extractors returns a list of registered Extractor names.
+	Extractors() []string
 	// AddExtractor adds an extractor to the data source. Multiple
 	// Extractors can be registered.
 	// The extractor will be called whenever the DataSource might
diff --git a/pkg/epp/datalayer/endpoint.go b/pkg/epp/datalayer/endpoint.go
index 74c1190..2d262eb 100644
--- a/pkg/epp/datalayer/endpoint.go
+++ b/pkg/epp/datalayer/endpoint.go
@@ -25,6 +25,7 @@ import (
 type EndpointPodState interface {
 	GetPod() *PodInfo
 	UpdatePod(*PodInfo)
+	GetAttributes() *Attributes
 }
 
 // EndpointMetricsState allows management of the Metrics related attributes.
@@ -48,11 +49,20 @@ type ModelServer struct {
 	attributes *Attributes
 }
 
-// NewEndpoint return a new (uninitialized) ModelServer.
-func NewEndpoint() *ModelServer {
-	return &ModelServer{
+// NewEndpoint returns a new ModelServer with the given PodInfo and Metrics.
+func NewEndpoint(pod *PodInfo, metrics *Metrics) *ModelServer {
+	if pod == nil {
+		pod = &PodInfo{}
+	}
+	if metrics == nil {
+		metrics = NewMetrics()
+	}
+	ep := &ModelServer{
 		attributes: NewAttributes(),
 	}
+	ep.UpdatePod(pod)
+	ep.UpdateMetrics(metrics)
+	return ep
 }
 
 // String returns a representation of the ModelServer. For brevity, only names of
@@ -89,6 +99,10 @@ func (srv *ModelServer) Keys() []string {
 	return srv.attributes.Keys()
 }
 
+func (srv *ModelServer) GetAttributes() *Attributes {
+	return srv.attributes
+}
+
 func (srv *ModelServer) Clone() *ModelServer {
 	clone := &ModelServer{
 		attributes: srv.attributes.Clone(),
diff --git a/pkg/epp/datalayer/factory.go b/pkg/epp/datalayer/factory.go
index 989527c..58da604 100644
--- a/pkg/epp/datalayer/factory.go
+++ b/pkg/epp/datalayer/factory.go
@@ -27,6 +27,10 @@ import (
 	v1 "sigs.k8s.io/gateway-api-inference-extension/api/v1"
 )
 
+const (
+	FeatureGate = "dataLayer"
+)
+
 // PoolInfo represents the DataStore information needed for endpoints.
 // TODO:
 // Consider if to remove/simplify in follow-ups. This is mostly for backward
@@ -78,9 +82,8 @@ func (lc *EndpointLifecycle) NewEndpoint(parent context.Context, inpod *PodInfo,
 		return nil
 	}
 
-	endpoint := NewEndpoint()
-	endpoint.UpdatePod(inpod)
-	collector := NewCollector() // for full backward compatibility, set the logger and poolinfo
+	endpoint := NewEndpoint(inpod, nil)
+	collector := NewCollector() // TODO or full backward compatibility, set the logger and poolinfo
 
 	if _, loaded := lc.collectors.LoadOrStore(key, collector); loaded {
 		// another goroutine already created and stored a collector for this endpoint.
diff --git a/pkg/epp/datalayer/metrics/datasource.go b/pkg/epp/datalayer/metrics/datasource.go
index 1e14d1b..81723d4 100644
--- a/pkg/epp/datalayer/metrics/datasource.go
+++ b/pkg/epp/datalayer/metrics/datasource.go
@@ -71,6 +71,18 @@ func (dataSrc *DataSource) Name() string {
 	return DataSourceName
 }
 
+// Extractors returns a list of registered Extractor names.
+func (dataSrc *DataSource) Extractors() []string {
+	names := []string{}
+	dataSrc.extractors.Range(func(_, val any) bool {
+		if ex, ok := val.(datalayer.Extractor); ok {
+			names = append(names, ex.Name())
+		}
+		return true // continue iteration
+	})
+	return names
+}
+
 // AddExtractor adds an extractor to the data source, validating it can process
 // the metrics' data source output type.
 func (dataSrc *DataSource) AddExtractor(extractor datalayer.Extractor) error {
diff --git a/pkg/epp/datalayer/metrics/extractor.go b/pkg/epp/datalayer/metrics/extractor.go
index f6142c4..27b1e07 100644
--- a/pkg/epp/datalayer/metrics/extractor.go
+++ b/pkg/epp/datalayer/metrics/extractor.go
@@ -26,9 +26,11 @@ import (
 	"time"
 
 	dto "github.com/prometheus/client_model/go"
+	"sigs.k8s.io/controller-runtime/pkg/log"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics"
+	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
 )
 
 const (
@@ -136,8 +138,10 @@ func (ext *Extractor) Extract(ctx context.Context, data any, ep datalayer.Endpoi
 		}
 	}
 
+	logger := log.FromContext(ctx).WithValues("pod", ep.GetPod().NamespacedName)
 	if updated {
 		clone.UpdateTime = time.Now()
+		logger.V(logutil.TRACE).Info("Refreshed metrics", "updated", clone)
 		ep.UpdateMetrics(clone)
 	}
 
diff --git a/pkg/epp/datalayer/metrics/logger.go b/pkg/epp/datalayer/metrics/logger.go
index fac757d..130c62f 100644
--- a/pkg/epp/datalayer/metrics/logger.go
+++ b/pkg/epp/datalayer/metrics/logger.go
@@ -18,6 +18,7 @@ package metrics
 
 import (
 	"context"
+	"fmt"
 	"time"
 
 	"github.com/go-logr/logr"
@@ -95,8 +96,8 @@ func printDebugMetrics(logger logr.Logger, datastore datalayer.PoolInfo, stalene
 	freshPods := datastore.PodList(podsWithFreshMetrics(stalenessThreshold))
 	stalePods := datastore.PodList(podsWithStaleMetrics(stalenessThreshold))
 
-	logger.V(logutil.VERBOSE).Info("Current Pods and metrics gathered",
-		"Fresh metrics", freshPods, "Stale metrics", stalePods)
+	logger.V(logutil.TRACE).Info("Current Pods and metrics gathered",
+		"Fresh metrics", fmt.Sprintf("%+v", freshPods), "Stale metrics", fmt.Sprintf("%+v", stalePods))
 }
 
 func refreshPrometheusMetrics(logger logr.Logger, datastore datalayer.PoolInfo, stalenessThreshold time.Duration) {
diff --git a/pkg/epp/datalayer/metrics/spec.go b/pkg/epp/datalayer/metrics/spec.go
index a079c13..e2fd98f 100644
--- a/pkg/epp/datalayer/metrics/spec.go
+++ b/pkg/epp/datalayer/metrics/spec.go
@@ -147,6 +147,9 @@ func (spec *Spec) labelsMatch(metricLabels []*dto.LabelPair) bool {
 // extractValue gets the numeric value from different metric types.
 // Currently only Gauge and Counter are supported.
 func extractValue(metric *dto.Metric) float64 {
+	if metric == nil {
+		return 0
+	}
 	if gauge := metric.GetGauge(); gauge != nil {
 		return gauge.GetValue()
 	}
diff --git a/pkg/epp/datastore/datastore.go b/pkg/epp/datastore/datastore.go
index dade694..5d6ff75 100644
--- a/pkg/epp/datastore/datastore.go
+++ b/pkg/epp/datastore/datastore.go
@@ -87,7 +87,7 @@ type datastore struct {
 	// poolAndObjectivesMu is used to synchronize access to pool and the objectives map.
 	poolAndObjectivesMu sync.RWMutex
 	pool                *v1.InferencePool
-	// key: InferenceObjective.Spec.ModelName, value: *InferenceObjective
+	// key: InferenceObjective name, value: *InferenceObjective
 	objectives map[string]*v1alpha2.InferenceObjective
 	// key: types.NamespacedName, value: backendmetrics.PodMetrics
 	pods *sync.Map
diff --git a/pkg/epp/flowcontrol/config.go b/pkg/epp/flowcontrol/config.go
index edc23ab..9c3cc87 100644
--- a/pkg/epp/flowcontrol/config.go
+++ b/pkg/epp/flowcontrol/config.go
@@ -23,6 +23,10 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/registry"
 )
 
+const (
+	FeatureGate = "flowControl"
+)
+
 // Config is the top-level configuration for the entire flow control module.
 // It embeds the configurations for the controller and the registry, providing a single point of entry for validation
 // and initialization.
diff --git a/pkg/epp/flowcontrol/contracts/mocks/mocks.go b/pkg/epp/flowcontrol/contracts/mocks/mocks.go
index 10f093b..49205ee 100644
--- a/pkg/epp/flowcontrol/contracts/mocks/mocks.go
+++ b/pkg/epp/flowcontrol/contracts/mocks/mocks.go
@@ -149,9 +149,9 @@ type MockManagedQueue struct {
 	// RemoveFunc allows a test to completely override the default Remove behavior.
 	RemoveFunc func(handle types.QueueItemHandle) (types.QueueItemAccessor, error)
 	// CleanupFunc allows a test to completely override the default Cleanup behavior.
-	CleanupFunc func(predicate framework.PredicateFunc) ([]types.QueueItemAccessor, error)
+	CleanupFunc func(predicate framework.PredicateFunc) []types.QueueItemAccessor
 	// DrainFunc allows a test to completely override the default Drain behavior.
-	DrainFunc func() ([]types.QueueItemAccessor, error)
+	DrainFunc func() []types.QueueItemAccessor
 
 	// mu protects access to the internal `items` map.
 	mu       sync.Mutex
@@ -209,7 +209,7 @@ func (m *MockManagedQueue) Remove(handle types.QueueItemHandle) (types.QueueItem
 }
 
 // Cleanup removes items matching a predicate. It checks for a test override before locking.
-func (m *MockManagedQueue) Cleanup(predicate framework.PredicateFunc) ([]types.QueueItemAccessor, error) {
+func (m *MockManagedQueue) Cleanup(predicate framework.PredicateFunc) []types.QueueItemAccessor {
 	if m.CleanupFunc != nil {
 		return m.CleanupFunc(predicate)
 	}
@@ -223,11 +223,11 @@ func (m *MockManagedQueue) Cleanup(predicate framework.PredicateFunc) ([]types.Q
 			delete(m.items, handle)
 		}
 	}
-	return removed, nil
+	return removed
 }
 
 // Drain removes all items from the queue. It checks for a test override before locking.
-func (m *MockManagedQueue) Drain() ([]types.QueueItemAccessor, error) {
+func (m *MockManagedQueue) Drain() []types.QueueItemAccessor {
 	if m.DrainFunc != nil {
 		return m.DrainFunc()
 	}
@@ -239,7 +239,7 @@ func (m *MockManagedQueue) Drain() ([]types.QueueItemAccessor, error) {
 		drained = append(drained, item)
 	}
 	m.items = make(map[types.QueueItemHandle]types.QueueItemAccessor)
-	return drained, nil
+	return drained
 }
 
 func (m *MockManagedQueue) FlowKey() types.FlowKey                    { return m.FlowKeyV }
@@ -268,17 +268,17 @@ func (m *MockManagedQueue) ByteSize() uint64 {
 }
 
 // PeekHead returns the first item found in the mock queue. Note: map iteration order is not guaranteed.
-func (m *MockManagedQueue) PeekHead() (types.QueueItemAccessor, error) {
+func (m *MockManagedQueue) PeekHead() types.QueueItemAccessor {
 	m.mu.Lock()
 	defer m.mu.Unlock()
 	m.init()
 	for _, item := range m.items {
-		return item, nil // Return first item found
+		return item // Return first item found
 	}
-	return nil, nil // Queue is empty
+	return nil // Queue is empty
 }
 
 // PeekTail is not implemented for this mock.
-func (m *MockManagedQueue) PeekTail() (types.QueueItemAccessor, error) {
-	return nil, nil
+func (m *MockManagedQueue) PeekTail() types.QueueItemAccessor {
+	return nil
 }
diff --git a/pkg/epp/flowcontrol/contracts/registry.go b/pkg/epp/flowcontrol/contracts/registry.go
index fe0b790..addc2c1 100644
--- a/pkg/epp/flowcontrol/contracts/registry.go
+++ b/pkg/epp/flowcontrol/contracts/registry.go
@@ -142,17 +142,24 @@ type RegistryShard interface {
 }
 
 // ManagedQueue defines the interface for a flow's queue on a specific shard.
-// It acts as a stateful decorator around an underlying `framework.SafeQueue`, augmenting it with statistics tracking.
+// It acts as a stateful decorator that *use an underlying framework.SafeQueue, augmenting it with statistics tracking
+// and lifecycle awareness (e.g., rejecting adds when a shard is draining).
 //
-// # Conformance
-//
-//   - Implementations MUST be goroutine-safe.
-//   - All mutating methods MUST ensure that the underlying queue state and the public statistics (`Len`, `ByteSize`)
-//     are updated as a single atomic transaction.
-//   - The `Add` method MUST return an error wrapping `ErrShardDraining` if the queue instance belongs to a parent shard
-//     that is no longer Active.
+// Conformance: Implementations MUST be goroutine-safe.
 type ManagedQueue interface {
-	framework.SafeQueue
+	// Add attempts to enqueue an item, performing an atomic check on the parent shard's lifecycle state before adding
+	// the item to the underlying queue.
+	// Returns ErrShardDraining if the parent shard is no longer Active.
+	Add(item types.QueueItemAccessor) error
+
+	// Remove atomically finds and removes an item from the underlying queue using its handle.
+	Remove(handle types.QueueItemHandle) (types.QueueItemAccessor, error)
+
+	// Cleanup removes all items from the underlying queue that satisfy the predicate.
+	Cleanup(predicate framework.PredicateFunc) []types.QueueItemAccessor
+
+	// Drain removes all items from the underlying queue.
+	Drain() []types.QueueItemAccessor
 
 	// FlowQueueAccessor returns a read-only, flow-aware accessor for this queue, used by policy plugins.
 	// Conformance: This method MUST NOT return nil.
diff --git a/pkg/epp/flowcontrol/controller/controller.go b/pkg/epp/flowcontrol/controller/controller.go
index a11ef26..751a420 100644
--- a/pkg/epp/flowcontrol/controller/controller.go
+++ b/pkg/epp/flowcontrol/controller/controller.go
@@ -374,7 +374,7 @@ func (fc *FlowController) selectDistributionCandidates(key types.FlowKey) ([]can
 					"flowKey", key, "shardID", shard.ID())
 				continue
 			}
-			candidates = append(candidates, candidate{worker.processor, shard.ID(), mq.ByteSize()})
+			candidates = append(candidates, candidate{worker.processor, shard.ID(), mq.FlowQueueAccessor().ByteSize()})
 		}
 		return nil
 	})
diff --git a/pkg/epp/flowcontrol/controller/internal/processor.go b/pkg/epp/flowcontrol/controller/internal/processor.go
index 2370fd6..489f421 100644
--- a/pkg/epp/flowcontrol/controller/internal/processor.go
+++ b/pkg/epp/flowcontrol/controller/internal/processor.go
@@ -20,7 +20,6 @@ import (
 	"context"
 	"errors"
 	"fmt"
-	"runtime"
 	"sync"
 	"sync/atomic"
 	"time"
@@ -165,6 +164,10 @@ func (sp *ShardProcessor) Run(ctx context.Context) {
 	sp.wg.Add(1)
 	go sp.runCleanupSweep(ctx)
 
+	// Create a ticker for periodic dispatch attempts to avoid tight loops
+	dispatchTicker := sp.clock.NewTicker(time.Millisecond)
+	defer dispatchTicker.Stop()
+
 	// This is the main worker loop. It continuously processes incoming requests and dispatches queued requests until the
 	// context is cancelled. The `select` statement has three cases:
 	//
@@ -172,9 +175,8 @@ func (sp *ShardProcessor) Run(ctx context.Context) {
 	//     loop will drain all queues and exit. This is the primary exit condition.
 	//  2. New Item Arrival: If an item is available on `enqueueChan`, it will be processed. This ensures that the
 	//     processor is responsive to new work.
-	//  3. Default (Dispatch): If neither of the above cases is ready, the `default` case executes, ensuring the loop is
-	//     non-blocking. It continuously attempts to dispatch items from the existing backlog, preventing starvation and
-	//     ensuring queues are drained.
+	//  3. Dispatch Ticker: Periodically triggers a dispatch cycle to attempt to dispatch items from existing queues,
+	//     ensuring that queued work is processed even when no new items arrive.
 	for {
 		select {
 		case <-ctx.Done():
@@ -193,14 +195,9 @@ func (sp *ShardProcessor) Run(ctx context.Context) {
 				continue
 			}
 			sp.enqueue(item)
-			sp.dispatchCycle(ctx)
-		default:
-			// If no new items are arriving, continuously try to dispatch from the backlog.
-			if !sp.dispatchCycle(ctx) {
-				// If no work was done, yield to the scheduler to prevent a tight, busy-loop when idle, while still allowing for
-				// immediate rescheduling.
-				runtime.Gosched()
-			}
+			sp.dispatchCycle(ctx) // Process immediately when an item arrives
+		case <-dispatchTicker.C():
+			sp.dispatchCycle(ctx) // Periodically attempt to dispatch from queues
 		}
 	}
 }
@@ -408,10 +405,7 @@ func (sp *ShardProcessor) sweepFinalizedItems() {
 		predicate := func(itemAcc types.QueueItemAccessor) bool {
 			return itemAcc.(*FlowItem).FinalState() != nil
 		}
-		removedItems, err := managedQ.Cleanup(predicate)
-		if err != nil {
-			logger.Error(err, "Error during ManagedQueue Cleanup", "flowKey", key)
-		}
+		removedItems := managedQ.Cleanup(predicate)
 		logger.V(logutil.DEBUG).Info("Swept finalized items and released capacity.",
 			"flowKey", key, "count", len(removedItems))
 	}
@@ -449,10 +443,7 @@ func (sp *ShardProcessor) shutdown() {
 func (sp *ShardProcessor) evictAll() {
 	processFn := func(managedQ contracts.ManagedQueue, logger logr.Logger) {
 		key := managedQ.FlowQueueAccessor().FlowKey()
-		removedItems, err := managedQ.Drain()
-		if err != nil {
-			logger.Error(err, "Error during ManagedQueue Drain", "flowKey", key)
-		}
+		removedItems := managedQ.Drain()
 
 		outcome := types.QueueOutcomeEvictedOther
 		errShutdown := fmt.Errorf("%w: %w", types.ErrEvicted, types.ErrFlowControllerNotRunning)
diff --git a/pkg/epp/flowcontrol/framework/errors.go b/pkg/epp/flowcontrol/framework/errors.go
index fb19c4a..79d62ac 100644
--- a/pkg/epp/flowcontrol/framework/errors.go
+++ b/pkg/epp/flowcontrol/framework/errors.go
@@ -26,13 +26,6 @@ import (
 // and might be handled or wrapped by the `contracts.FlowRegistry`'s `contracts.ManagedQueue` or the
 // `controller.FlowController`.
 var (
-	// ErrNilQueueItem indicates that a nil `types.QueueItemAccessor` was passed to `SafeQueue.Add()`.
-	ErrNilQueueItem = errors.New("queue item cannot be nil")
-
-	// ErrQueueEmpty indicates an attempt to perform an operation on an empty `SafeQueue` that requires one or more items
-	// (e.g., calling `SafeQueue.PeekHead()`).
-	ErrQueueEmpty = errors.New("queue is empty")
-
 	// ErrInvalidQueueItemHandle indicates that a `types.QueueItemHandle` provided to a `SafeQueue` operation (e.g.,
 	// `SafeQueue.Remove()`) is not valid for that queue, has been invalidated, or does not correspond to an actual item
 	// in the queue.
diff --git a/pkg/epp/flowcontrol/framework/mocks/mocks.go b/pkg/epp/flowcontrol/framework/mocks/mocks.go
index ff8441f..29eead4 100644
--- a/pkg/epp/flowcontrol/framework/mocks/mocks.go
+++ b/pkg/epp/flowcontrol/framework/mocks/mocks.go
@@ -38,9 +38,7 @@ type MockFlowQueueAccessor struct {
 	LenV          int
 	ByteSizeV     uint64
 	PeekHeadV     types.QueueItemAccessor
-	PeekHeadErrV  error
 	PeekTailV     types.QueueItemAccessor
-	PeekTailErrV  error
 	FlowKeyV      types.FlowKey
 	ComparatorV   framework.ItemComparator
 	CapabilitiesV []framework.QueueCapability
@@ -53,12 +51,12 @@ func (m *MockFlowQueueAccessor) Comparator() framework.ItemComparator      { ret
 func (m *MockFlowQueueAccessor) FlowKey() types.FlowKey                    { return m.FlowKeyV }
 func (m *MockFlowQueueAccessor) Capabilities() []framework.QueueCapability { return m.CapabilitiesV }
 
-func (m *MockFlowQueueAccessor) PeekHead() (types.QueueItemAccessor, error) {
-	return m.PeekHeadV, m.PeekHeadErrV
+func (m *MockFlowQueueAccessor) PeekHead() types.QueueItemAccessor {
+	return m.PeekHeadV
 }
 
-func (m *MockFlowQueueAccessor) PeekTail() (types.QueueItemAccessor, error) {
-	return m.PeekTailV, m.PeekTailErrV
+func (m *MockFlowQueueAccessor) PeekTail() types.QueueItemAccessor {
+	return m.PeekTailV
 }
 
 var _ framework.FlowQueueAccessor = &MockFlowQueueAccessor{}
@@ -108,13 +106,11 @@ type MockSafeQueue struct {
 	LenV          int
 	ByteSizeV     uint64
 	PeekHeadV     types.QueueItemAccessor
-	PeekHeadErrV  error
 	PeekTailV     types.QueueItemAccessor
-	PeekTailErrV  error
-	AddFunc       func(item types.QueueItemAccessor) error
+	AddFunc       func(item types.QueueItemAccessor)
 	RemoveFunc    func(handle types.QueueItemHandle) (types.QueueItemAccessor, error)
-	CleanupFunc   func(predicate framework.PredicateFunc) ([]types.QueueItemAccessor, error)
-	DrainFunc     func() ([]types.QueueItemAccessor, error)
+	CleanupFunc   func(predicate framework.PredicateFunc) []types.QueueItemAccessor
+	DrainFunc     func() []types.QueueItemAccessor
 }
 
 func (m *MockSafeQueue) Name() string                              { return m.NameV }
@@ -122,19 +118,18 @@ func (m *MockSafeQueue) Capabilities() []framework.QueueCapability { return m.Ca
 func (m *MockSafeQueue) Len() int                                  { return m.LenV }
 func (m *MockSafeQueue) ByteSize() uint64                          { return m.ByteSizeV }
 
-func (m *MockSafeQueue) PeekHead() (types.QueueItemAccessor, error) {
-	return m.PeekHeadV, m.PeekHeadErrV
+func (m *MockSafeQueue) PeekHead() types.QueueItemAccessor {
+	return m.PeekHeadV
 }
 
-func (m *MockSafeQueue) PeekTail() (types.QueueItemAccessor, error) {
-	return m.PeekTailV, m.PeekTailErrV
+func (m *MockSafeQueue) PeekTail() types.QueueItemAccessor {
+	return m.PeekTailV
 }
 
-func (m *MockSafeQueue) Add(item types.QueueItemAccessor) error {
+func (m *MockSafeQueue) Add(item types.QueueItemAccessor) {
 	if m.AddFunc != nil {
-		return m.AddFunc(item)
+		m.AddFunc(item)
 	}
-	return nil
 }
 
 func (m *MockSafeQueue) Remove(handle types.QueueItemHandle) (types.QueueItemAccessor, error) {
@@ -144,18 +139,18 @@ func (m *MockSafeQueue) Remove(handle types.QueueItemHandle) (types.QueueItemAcc
 	return nil, nil
 }
 
-func (m *MockSafeQueue) Cleanup(predicate framework.PredicateFunc) ([]types.QueueItemAccessor, error) {
+func (m *MockSafeQueue) Cleanup(predicate framework.PredicateFunc) []types.QueueItemAccessor {
 	if m.CleanupFunc != nil {
 		return m.CleanupFunc(predicate)
 	}
-	return nil, nil
+	return nil
 }
 
-func (m *MockSafeQueue) Drain() ([]types.QueueItemAccessor, error) {
+func (m *MockSafeQueue) Drain() []types.QueueItemAccessor {
 	if m.DrainFunc != nil {
 		return m.DrainFunc()
 	}
-	return nil, nil
+	return nil
 }
 
 var _ framework.SafeQueue = &MockSafeQueue{}
diff --git a/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead.go b/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead.go
index 41033e7..ae4cbd6 100644
--- a/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead.go
+++ b/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead.go
@@ -68,8 +68,8 @@ func (p *bestHead) SelectQueue(band framework.PriorityBandAccessor) (framework.F
 			return true
 		}
 
-		item, err := queue.PeekHead()
-		if err != nil || item == nil {
+		item := queue.PeekHead()
+		if item == nil {
 			return true
 		}
 
diff --git a/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/fcfs/fcfs.go b/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/fcfs/fcfs.go
index 7addb9d..63ffe31 100644
--- a/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/fcfs/fcfs.go
+++ b/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/fcfs/fcfs.go
@@ -18,8 +18,6 @@ limitations under the License.
 package fcfs
 
 import (
-	"errors"
-
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/types"
@@ -86,11 +84,7 @@ func (p *fcfs) SelectItem(queue framework.FlowQueueAccessor) (types.QueueItemAcc
 	if queue == nil {
 		return nil, nil
 	}
-	item, err := queue.PeekHead()
-	if errors.Is(err, framework.ErrQueueEmpty) {
-		return nil, nil
-	}
-	return item, err
+	return queue.PeekHead(), nil
 }
 
 // Comparator returns a `framework.ItemComparator` based on enqueue time.
diff --git a/pkg/epp/flowcontrol/framework/plugins/queue/listqueue.go b/pkg/epp/flowcontrol/framework/plugins/queue/listqueue.go
index 082510e..0d241cf 100644
--- a/pkg/epp/flowcontrol/framework/plugins/queue/listqueue.go
+++ b/pkg/epp/flowcontrol/framework/plugins/queue/listqueue.go
@@ -102,18 +102,13 @@ func newListQueue() *listQueue {
 // --- `framework.SafeQueue` Interface Implementation ---
 
 // Add enqueues an item to the back of the list.
-func (lq *listQueue) Add(item types.QueueItemAccessor) error {
+func (lq *listQueue) Add(item types.QueueItemAccessor) {
 	lq.mu.Lock()
 	defer lq.mu.Unlock()
 
-	if item == nil {
-		return framework.ErrNilQueueItem
-	}
-
 	element := lq.requests.PushBack(item)
 	lq.byteSize.Add(item.OriginalRequest().ByteSize())
 	item.SetHandle(&listItemHandle{element: element, owner: lq})
-	return nil
 }
 
 // Remove removes an item identified by the given handle from the queue.
@@ -142,7 +137,7 @@ func (lq *listQueue) Remove(handle types.QueueItemHandle) (types.QueueItemAccess
 }
 
 // Cleanup removes items from the queue that satisfy the predicate.
-func (lq *listQueue) Cleanup(predicate framework.PredicateFunc) (cleanedItems []types.QueueItemAccessor, err error) {
+func (lq *listQueue) Cleanup(predicate framework.PredicateFunc) (cleanedItems []types.QueueItemAccessor) {
 	lq.mu.Lock()
 	defer lq.mu.Unlock()
 
@@ -162,11 +157,11 @@ func (lq *listQueue) Cleanup(predicate framework.PredicateFunc) (cleanedItems []
 			removedItems = append(removedItems, item)
 		}
 	}
-	return removedItems, nil
+	return removedItems
 }
 
 // Drain removes all items from the queue and returns them.
-func (lq *listQueue) Drain() (removedItems []types.QueueItemAccessor, err error) {
+func (lq *listQueue) Drain() (removedItems []types.QueueItemAccessor) {
 	lq.mu.Lock()
 	defer lq.mu.Unlock()
 
@@ -182,7 +177,7 @@ func (lq *listQueue) Drain() (removedItems []types.QueueItemAccessor, err error)
 
 	lq.requests.Init()
 	lq.byteSize.Store(0)
-	return removedItems, nil
+	return removedItems
 }
 
 // Name returns the name of the queue.
@@ -208,25 +203,25 @@ func (lq *listQueue) ByteSize() uint64 {
 }
 
 // PeekHead returns the item at the front of the queue without removing it.
-func (lq *listQueue) PeekHead() (head types.QueueItemAccessor, err error) {
+func (lq *listQueue) PeekHead() types.QueueItemAccessor {
 	lq.mu.RLock()
 	defer lq.mu.RUnlock()
 
 	if lq.requests.Len() == 0 {
-		return nil, framework.ErrQueueEmpty
+		return nil
 	}
 	element := lq.requests.Front()
-	return element.Value.(types.QueueItemAccessor), nil
+	return element.Value.(types.QueueItemAccessor)
 }
 
 // PeekTail returns the item at the back of the queue without removing it.
-func (lq *listQueue) PeekTail() (tail types.QueueItemAccessor, err error) {
+func (lq *listQueue) PeekTail() types.QueueItemAccessor {
 	lq.mu.RLock()
 	defer lq.mu.RUnlock()
 
 	if lq.requests.Len() == 0 {
-		return nil, framework.ErrQueueEmpty
+		return nil
 	}
 	element := lq.requests.Back()
-	return element.Value.(types.QueueItemAccessor), nil
+	return element.Value.(types.QueueItemAccessor)
 }
diff --git a/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap.go b/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap.go
index 41a15fa..4a3bc87 100644
--- a/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap.go
+++ b/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap.go
@@ -115,56 +115,51 @@ func (h *maxMinHeap) ByteSize() uint64 {
 
 // PeekHead returns the item with the highest priority (max value) without removing it.
 // Time complexity: O(1).
-func (h *maxMinHeap) PeekHead() (types.QueueItemAccessor, error) {
+func (h *maxMinHeap) PeekHead() types.QueueItemAccessor {
 	h.mu.RLock()
 	defer h.mu.RUnlock()
 
 	if len(h.items) == 0 {
-		return nil, framework.ErrQueueEmpty
+		return nil
 	}
 	// The root of the max-min heap is always the maximum element.
-	return h.items[0], nil
+	return h.items[0]
 }
 
 // PeekTail returns the item with the lowest priority (min value) without removing it.
 // Time complexity: O(1).
-func (h *maxMinHeap) PeekTail() (types.QueueItemAccessor, error) {
+func (h *maxMinHeap) PeekTail() types.QueueItemAccessor {
 	h.mu.RLock()
 	defer h.mu.RUnlock()
 
 	n := len(h.items)
 	if n == 0 {
-		return nil, framework.ErrQueueEmpty
+		return nil
 	}
 	if n == 1 {
-		return h.items[0], nil
+		return h.items[0]
 	}
 	if n == 2 {
 		// With two items, the root is max, the second is min.
-		return h.items[1], nil
+		return h.items[1]
 	}
 
 	// With three or more items, the minimum element is guaranteed to be one of the two children of the root (at indices 1
 	// and 2). We must compare them to find the true minimum.
 	if h.comparator.Func()(h.items[1], h.items[2]) {
-		return h.items[2], nil
+		return h.items[2]
 	}
-	return h.items[1], nil
+	return h.items[1]
 }
 
 // Add adds an item to the queue.
 // Time complexity: O(log n).
-func (h *maxMinHeap) Add(item types.QueueItemAccessor) error {
+func (h *maxMinHeap) Add(item types.QueueItemAccessor) {
 	h.mu.Lock()
 	defer h.mu.Unlock()
 
-	if item == nil {
-		return framework.ErrNilQueueItem
-	}
-
 	h.push(item)
 	h.byteSize.Add(item.OriginalRequest().ByteSize())
-	return nil
 }
 
 // push adds an item to the heap and restores the heap property.
@@ -425,7 +420,7 @@ func isMinLevel(i int) bool {
 }
 
 // Cleanup removes items from the queue that satisfy the predicate.
-func (h *maxMinHeap) Cleanup(predicate framework.PredicateFunc) ([]types.QueueItemAccessor, error) {
+func (h *maxMinHeap) Cleanup(predicate framework.PredicateFunc) []types.QueueItemAccessor {
 	h.mu.Lock()
 	defer h.mu.Unlock()
 
@@ -459,11 +454,11 @@ func (h *maxMinHeap) Cleanup(predicate framework.PredicateFunc) ([]types.QueueIt
 		}
 	}
 
-	return removedItems, nil
+	return removedItems
 }
 
 // Drain removes all items from the queue.
-func (h *maxMinHeap) Drain() ([]types.QueueItemAccessor, error) {
+func (h *maxMinHeap) Drain() []types.QueueItemAccessor {
 	h.mu.Lock()
 	defer h.mu.Unlock()
 
@@ -482,5 +477,5 @@ func (h *maxMinHeap) Drain() ([]types.QueueItemAccessor, error) {
 	h.handles = make(map[types.QueueItemHandle]*heapItem)
 	h.byteSize.Store(0)
 
-	return drainedItems, nil
+	return drainedItems
 }
diff --git a/pkg/epp/flowcontrol/framework/queue.go b/pkg/epp/flowcontrol/framework/queue.go
index d00cccf..3f33df7 100644
--- a/pkg/epp/flowcontrol/framework/queue.go
+++ b/pkg/epp/flowcontrol/framework/queue.go
@@ -20,30 +20,23 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/types"
 )
 
-// QueueCapability defines a functional capability that a `SafeQueue` implementation can provide.
-// These capabilities allow policies (like `IntraFlowDispatchPolicy`) to declare their operational requirements,
-// ensuring that a policy is always paired with a compatible queue.
-//
-// For example, a policy that requires a priority-ordered queue would declare `CapabilityPriorityConfigurable`, and the
-// `contracts.FlowRegistry` would ensure it is paired with a queue implementation (like a heap) that provides this
-// capability.
-//
-// While a simpler boolean method (e.g., `IsPriorityConfigurable()`) could satisfy current needs, this slice-based
-// approach is intentionally chosen for future extensibility. It allows a queue to advertise multiple features (e.g.,
-// dynamic priority reshuffling, size bounds, etc.) without future breaking changes to the `SafeQueue` interface.
+// QueueCapability defines a functional capability that a SafeQueue implementation can provide.
+// These capabilities allow policies to declare their operational requirements, ensuring that a policy is always paired
+// with a compatible queue.
 type QueueCapability string
 
 const (
 	// CapabilityFIFO indicates that the queue operates in a First-In, First-Out manner.
-	// `PeekHead()` will return the oldest item (by logical enqueue time).
+	// PeekHead() will return the oldest item (by logical enqueue time).
 	CapabilityFIFO QueueCapability = "FIFO"
 
-	// CapabilityPriorityConfigurable indicates that the queue's ordering is determined by an `ItemComparator`.
-	// `PeekHead()` will return the highest priority item according to this comparator.
+	// CapabilityPriorityConfigurable indicates that the queue's ordering is determined by an ItemComparator.
+	// PeekHead() will return the highest priority item, and PeekTail() will return the lowest priority item according to
+	// this comparator.
 	CapabilityPriorityConfigurable QueueCapability = "PriorityConfigurable"
 )
 
-// QueueInspectionMethods defines `SafeQueue`'s read-only methods.
+// QueueInspectionMethods defines SafeQueue's read-only methods.
 type QueueInspectionMethods interface {
 	// Name returns a string identifier for the concrete queue implementation type (e.g., "ListQueue").
 	Name() string
@@ -59,48 +52,44 @@ type QueueInspectionMethods interface {
 
 	// PeekHead returns the item at the "head" of the queue (the item with the highest priority according to the queue's
 	// ordering) without removing it.
-	//
-	// Returns `ErrQueueEmpty` if the queue is empty.
-	PeekHead() (peekedItem types.QueueItemAccessor, err error)
+	// Returns nil if the queue is empty.
+	PeekHead() types.QueueItemAccessor
 
 	// PeekTail returns the item at the "tail" of the queue (the item with the lowest priority according to the queue's
 	// ordering) without removing it.
-	//
-	// Returns `ErrQueueEmpty` if the queue is empty.
-	PeekTail() (peekedItem types.QueueItemAccessor, err error)
+	// Returns nil if the queue is empty.
+	PeekTail() types.QueueItemAccessor
 }
 
 // PredicateFunc defines a function that returns true if a given item matches a certain condition.
-// It is used by `SafeQueue.Cleanup` to filter items.
+// It is used by SafeQueue.Cleanup to filter items.
 type PredicateFunc func(item types.QueueItemAccessor) bool
 
 // SafeQueue defines the contract for a single, concurrent-safe queue implementation.
-//
+// This interface is designed for in-memory, synchronous flow control. Implementations are expected to be unbounded;
+// capacity management occurs outside the queue implementation.
 // All implementations MUST be goroutine-safe.
 type SafeQueue interface {
 	QueueInspectionMethods
 
-	// Add attempts to enqueue an item. On success, it must associate a new, unique `types.QueueItemHandle` with the item
-	// by calling `item.SetHandle()`.
-	Add(item types.QueueItemAccessor) error
+	// Add enqueues an item. It must associate a new, unique types.QueueItemHandle with the item by calling
+	// item.SetHandle().
+	// Contract: The caller MUST NOT provide a nil item.
+	Add(item types.QueueItemAccessor)
 
 	// Remove atomically finds and removes the item identified by the given handle.
-	//
-	// On success, implementations MUST invalidate the provided handle by calling `handle.Invalidate()`.
-	//
+	// On success, implementations MUST invalidate the provided handle by calling handle.Invalidate().
 	// Returns the removed item.
-	// Returns `ErrInvalidQueueItemHandle` if the handle is invalid (e.g., nil, wrong type, already invalidated).
-	// Returns `ErrQueueItemNotFound` if the handle is valid but the item is not in the queue.
+	// Returns ErrInvalidQueueItemHandle if the handle is invalid.
+	// Returns ErrQueueItemNotFound if the handle is valid but the item is not in the queue.
 	Remove(handle types.QueueItemHandle) (removedItem types.QueueItemAccessor, err error)
 
 	// Cleanup iterates through the queue and atomically removes all items for which the predicate returns true, returning
 	// them in a slice.
-	//
 	// The handle for each removed item MUST be invalidated.
-	Cleanup(predicate PredicateFunc) (cleanedItems []types.QueueItemAccessor, err error)
+	Cleanup(predicate PredicateFunc) (cleanedItems []types.QueueItemAccessor)
 
 	// Drain atomically removes all items from the queue and returns them in a slice.
-	//
 	// The handle for all removed items MUST be invalidated. The queue MUST be empty after this operation.
-	Drain() (drainedItems []types.QueueItemAccessor, err error)
+	Drain() (drainedItems []types.QueueItemAccessor)
 }
diff --git a/pkg/epp/flowcontrol/registry/managedqueue.go b/pkg/epp/flowcontrol/registry/managedqueue.go
index e56cdd8..b141dff 100644
--- a/pkg/epp/flowcontrol/registry/managedqueue.go
+++ b/pkg/epp/flowcontrol/registry/managedqueue.go
@@ -100,7 +100,7 @@ func newManagedQueue(
 		"flowKey", key,
 		"queueType", queue.Name(),
 	)
-	mq := &managedQueue{
+	return &managedQueue{
 		queue:          queue,
 		dispatchPolicy: dispatchPolicy,
 		key:            key,
@@ -108,37 +108,30 @@ func newManagedQueue(
 		logger:         mqLogger,
 		isDraining:     isDraining,
 	}
-	return mq
 }
 
 // FlowQueueAccessor returns a read-only, flow-aware view of this queue.
-// This accessor is primarily used by policy plugins to inspect the queue's state in a structured way.
 func (mq *managedQueue) FlowQueueAccessor() framework.FlowQueueAccessor {
 	return &flowQueueAccessor{mq: mq}
 }
 
-// Add wraps the underlying `framework.SafeQueue.Add` call and atomically updates the queue's and the parent shard's
-// statistics.
+// Add performs an atomic check on the parent shard's lifecycle state before adding the item to the underlying queue.
+// This is the critical enforcement point that prevents new requests from entering a draining shard.
 func (mq *managedQueue) Add(item types.QueueItemAccessor) error {
-	// Enforce the system's routing contract by rejecting new work for a Draining shard.
-	// This prevents a race where a caller could route a request to a shard just as it begins to drain.
-	if mq.isDraining() {
-		return contracts.ErrShardDraining
-	}
-
 	mq.mu.Lock()
 	defer mq.mu.Unlock()
 
-	if err := mq.queue.Add(item); err != nil {
-		return err
+	if mq.isDraining() {
+		return contracts.ErrShardDraining
 	}
+	mq.queue.Add(item)
+
 	mq.propagateStatsDeltaLocked(1, int64(item.OriginalRequest().ByteSize()))
 	mq.logger.V(logging.TRACE).Info("Request added to queue", "requestID", item.OriginalRequest().ID())
 	return nil
 }
 
-// Remove wraps the underlying `framework.SafeQueue.Remove` call and atomically updates statistics upon successful
-// removal.
+// Remove wraps the underlying framework.SafeQueue.Remove and updates statistics.
 func (mq *managedQueue) Remove(handle types.QueueItemHandle) (types.QueueItemAccessor, error) {
 	mq.mu.Lock()
 	defer mq.mu.Unlock()
@@ -152,52 +145,43 @@ func (mq *managedQueue) Remove(handle types.QueueItemHandle) (types.QueueItemAcc
 	return removedItem, nil
 }
 
-// Cleanup wraps the underlying `framework.SafeQueue.Cleanup` call and atomically updates statistics for all removed
-// items.
-func (mq *managedQueue) Cleanup(predicate framework.PredicateFunc) (cleanedItems []types.QueueItemAccessor, err error) {
+// Cleanup wraps the underlying framework.SafeQueue.Cleanup and updates statistics.
+func (mq *managedQueue) Cleanup(predicate framework.PredicateFunc) []types.QueueItemAccessor {
 	mq.mu.Lock()
 	defer mq.mu.Unlock()
 
-	cleanedItems, err = mq.queue.Cleanup(predicate)
-	if err != nil {
-		return nil, err
-	}
+	cleanedItems := mq.queue.Cleanup(predicate)
 	if len(cleanedItems) == 0 {
-		return cleanedItems, nil
+		return nil
 	}
 	mq.propagateStatsDeltaForRemovedItemsLocked(cleanedItems)
 	mq.logger.V(logging.DEBUG).Info("Cleaned up queue", "removedItemCount", len(cleanedItems))
-	return cleanedItems, nil
+	return cleanedItems
 }
 
-// Drain wraps the underlying `framework.SafeQueue.Drain` call and atomically updates statistics for all removed items.
-func (mq *managedQueue) Drain() ([]types.QueueItemAccessor, error) {
+// Drain wraps the underlying framework.SafeQueue.Drain and updates statistics.
+func (mq *managedQueue) Drain() []types.QueueItemAccessor {
 	mq.mu.Lock()
 	defer mq.mu.Unlock()
 
-	drainedItems, err := mq.queue.Drain()
-	if err != nil {
-		return nil, err
-	}
+	drainedItems := mq.queue.Drain()
 	if len(drainedItems) == 0 {
-		return drainedItems, nil
+		return nil
 	}
 	mq.propagateStatsDeltaForRemovedItemsLocked(drainedItems)
 	mq.logger.V(logging.DEBUG).Info("Drained queue", "itemCount", len(drainedItems))
-	return drainedItems, nil
+	return drainedItems
 }
 
-// --- Pass-through and Accessor Methods ---
-
-func (mq *managedQueue) Name() string                               { return mq.queue.Name() }
-func (mq *managedQueue) Capabilities() []framework.QueueCapability  { return mq.queue.Capabilities() }
-func (mq *managedQueue) Len() int                                   { return int(mq.len.Load()) }
-func (mq *managedQueue) ByteSize() uint64                           { return uint64(mq.byteSize.Load()) }
-func (mq *managedQueue) PeekHead() (types.QueueItemAccessor, error) { return mq.queue.PeekHead() }
-func (mq *managedQueue) PeekTail() (types.QueueItemAccessor, error) { return mq.queue.PeekTail() }
-func (mq *managedQueue) Comparator() framework.ItemComparator       { return mq.dispatchPolicy.Comparator() }
+// Len returns the current number of items in the queue.
+func (mq *managedQueue) Len() int {
+	return int(mq.len.Load())
+}
 
-// --- Internal Methods ---
+// ByteSize returns the current total byte size of all items in the queue.
+func (mq *managedQueue) ByteSize() uint64 {
+	return uint64(mq.byteSize.Load())
+}
 
 // propagateStatsDeltaLocked updates the queue's statistics and propagates the delta to the parent shard.
 // It must be called while holding the `managedQueue.mu` lock.
@@ -243,11 +227,18 @@ type flowQueueAccessor struct {
 
 var _ framework.FlowQueueAccessor = &flowQueueAccessor{}
 
-func (a *flowQueueAccessor) Name() string                               { return a.mq.Name() }
-func (a *flowQueueAccessor) Capabilities() []framework.QueueCapability  { return a.mq.Capabilities() }
-func (a *flowQueueAccessor) Len() int                                   { return a.mq.Len() }
-func (a *flowQueueAccessor) ByteSize() uint64                           { return a.mq.ByteSize() }
-func (a *flowQueueAccessor) PeekHead() (types.QueueItemAccessor, error) { return a.mq.PeekHead() }
-func (a *flowQueueAccessor) PeekTail() (types.QueueItemAccessor, error) { return a.mq.PeekTail() }
-func (a *flowQueueAccessor) Comparator() framework.ItemComparator       { return a.mq.Comparator() }
-func (a *flowQueueAccessor) FlowKey() types.FlowKey                     { return a.mq.key }
+// --- Read-only pass-through methods to the underlying SafeQueue ---
+func (a *flowQueueAccessor) Name() string { return a.mq.queue.Name() }
+func (a *flowQueueAccessor) Capabilities() []framework.QueueCapability {
+	return a.mq.queue.Capabilities()
+}
+func (a *flowQueueAccessor) PeekHead() types.QueueItemAccessor { return a.mq.queue.PeekHead() }
+func (a *flowQueueAccessor) PeekTail() types.QueueItemAccessor { return a.mq.queue.PeekTail() }
+
+// --- Read-only methods from the managedQueue wrapper ---
+func (a *flowQueueAccessor) Len() int         { return a.mq.Len() }
+func (a *flowQueueAccessor) ByteSize() uint64 { return a.mq.ByteSize() }
+func (a *flowQueueAccessor) Comparator() framework.ItemComparator {
+	return a.mq.dispatchPolicy.Comparator()
+}
+func (a *flowQueueAccessor) FlowKey() types.FlowKey { return a.mq.key }
diff --git a/pkg/epp/requestcontrol/director.go b/pkg/epp/requestcontrol/director.go
index f6f7dee..e8db59b 100644
--- a/pkg/epp/requestcontrol/director.go
+++ b/pkg/epp/requestcontrol/director.go
@@ -32,6 +32,7 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/handlers"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metadata"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics"
@@ -41,10 +42,15 @@ import (
 	requtil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/request"
 )
 
+const (
+	// TODO: Make these configurable per plugin via config.
+	prepareDataTimeout = 400 * time.Millisecond
+)
+
 // Datastore defines the interface required by the Director.
 type Datastore interface {
 	PoolGet() (*v1.InferencePool, error)
-	ObjectiveGet(modelName string) *v1alpha2.InferenceObjective
+	ObjectiveGet(objectiveName string) *v1alpha2.InferenceObjective
 	PodList(predicate func(backendmetrics.PodMetrics) bool) []backendmetrics.PodMetrics
 }
 
@@ -89,16 +95,28 @@ type Director struct {
 	defaultPriority int
 }
 
-// HandleRequest orchestrates the request lifecycle.
-// It always returns the requestContext even in the error case, as the request context is used in error handling.
-func (d *Director) HandleRequest(ctx context.Context, reqCtx *handlers.RequestContext) (*handlers.RequestContext, error) {
-	logger := log.FromContext(ctx)
+// getInferenceObjective fetches the inferenceObjective from the datastore otherwise creates a new one based on reqCtx.
+func (d *Director) getInferenceObjective(ctx context.Context, reqCtx *handlers.RequestContext) *v1alpha2.InferenceObjective {
+	infObjective := d.datastore.ObjectiveGet(reqCtx.ObjectiveKey)
+	if infObjective == nil {
+		log.FromContext(ctx).V(logutil.VERBOSE).Info("No associated InferenceObjective found, using default", "objectiveKey", reqCtx.ObjectiveKey)
+		infObjective = &v1alpha2.InferenceObjective{
+			Spec: v1alpha2.InferenceObjectiveSpec{
+				Priority: &d.defaultPriority,
+			},
+		}
+	} else if infObjective.Spec.Priority == nil {
+		// Default to 0 if not specified.
+		infObjective.Spec.Priority = &d.defaultPriority
+	}
+	return infObjective
+}
 
-	// Parse Request, Resolve Target Models, and Determine Parameters
+// resolveTargetModel is a helper to update reqCtx with target model based on request.
+func (d *Director) resolveTargetModel(reqCtx *handlers.RequestContext) (*handlers.RequestContext, error) {
 	requestBodyMap := reqCtx.Request.Body
 	var ok bool
 	reqCtx.IncomingModelName, ok = requestBodyMap["model"].(string)
-
 	if !ok {
 		return reqCtx, errutil.Error{Code: errutil.BadRequest, Msg: "model not found in request body"}
 	}
@@ -107,24 +125,28 @@ func (d *Director) HandleRequest(ctx context.Context, reqCtx *handlers.RequestCo
 		reqCtx.TargetModelName = reqCtx.IncomingModelName
 	}
 	reqCtx.Request.Body["model"] = reqCtx.TargetModelName
+	return reqCtx, nil
+}
 
+// HandleRequest orchestrates the request lifecycle.
+// It always returns the requestContext even in the error case, as the request context is used in error handling.
+func (d *Director) HandleRequest(ctx context.Context, reqCtx *handlers.RequestContext) (*handlers.RequestContext, error) {
+	logger := log.FromContext(ctx)
+
+	// Resolve target model and update req context.
+	reqCtx, err := d.resolveTargetModel(reqCtx)
+	if err != nil {
+		return reqCtx, err
+	}
+
+	// Parse request body.
 	requestBody, err := requtil.ExtractRequestBody(reqCtx.Request.Body)
 	if err != nil {
 		return reqCtx, errutil.Error{Code: errutil.BadRequest, Msg: fmt.Errorf("failed to extract request data: %w", err).Error()}
 	}
 
-	infObjective := d.datastore.ObjectiveGet(reqCtx.ObjectiveKey)
-	if infObjective == nil {
-		logger.V(logutil.VERBOSE).Info("No associated InferenceObjective found, using default", "objectiveKey", reqCtx.ObjectiveKey)
-		infObjective = &v1alpha2.InferenceObjective{
-			Spec: v1alpha2.InferenceObjectiveSpec{
-				Priority: &d.defaultPriority,
-			},
-		}
-	} else if infObjective.Spec.Priority == nil {
-		// Default to 0 if not specified.
-		infObjective.Spec.Priority = &d.defaultPriority
-	}
+	// Parse inference objective.
+	infObjective := d.getInferenceObjective(ctx, reqCtx)
 
 	// Prepare LLMRequest (needed for both saturation detection and Scheduler)
 	reqCtx.SchedulingRequest = &schedulingtypes.LLMRequest{
@@ -144,13 +166,25 @@ func (d *Director) HandleRequest(ctx context.Context, reqCtx *handlers.RequestCo
 	if len(candidatePods) == 0 {
 		return reqCtx, errutil.Error{Code: errutil.ServiceUnavailable, Msg: "failed to find candidate pods for serving the request"}
 	}
-
 	if err := d.admissionController.Admit(ctx, reqCtx, candidatePods, *infObjective.Spec.Priority); err != nil {
 		logger.V(logutil.DEFAULT).Info("Request rejected by admission control", "error", err)
 		return reqCtx, err
 	}
+	snapshotOfCandidatePods := d.toSchedulerPodMetrics(candidatePods)
 
-	result, err := d.scheduler.Schedule(ctx, reqCtx.SchedulingRequest, d.toSchedulerPodMetrics(candidatePods))
+	// Prepare per request data by running PrepareData plugins.
+	if d.runPrepareDataPlugins(ctx, reqCtx.SchedulingRequest, snapshotOfCandidatePods) != nil {
+		// Don't fail the request if PrepareData plugins fail.
+		logger.V(logutil.DEFAULT).Error(err, "failed to prepare per request data")
+	}
+
+	// Run admit request plugins
+	if !d.runAdmissionPlugins(ctx, reqCtx.SchedulingRequest, snapshotOfCandidatePods) {
+		logger.V(logutil.DEFAULT).Info("Request cannot be admitted")
+		return reqCtx, errutil.Error{Code: errutil.Internal, Msg: "request cannot be admitted"}
+	}
+
+	result, err := d.scheduler.Schedule(ctx, reqCtx.SchedulingRequest, snapshotOfCandidatePods)
 	if err != nil {
 		return reqCtx, errutil.Error{Code: errutil.InferencePoolResourceExhausted, Msg: fmt.Errorf("failed to find target pod: %w", err).Error()}
 	}
@@ -244,7 +278,11 @@ func (d *Director) prepareRequest(ctx context.Context, reqCtx *handlers.RequestC
 func (d *Director) toSchedulerPodMetrics(pods []backendmetrics.PodMetrics) []schedulingtypes.Pod {
 	pm := make([]schedulingtypes.Pod, len(pods))
 	for i, pod := range pods {
-		pm[i] = &schedulingtypes.PodMetrics{Pod: pod.GetPod().Clone(), MetricsState: pod.GetMetrics().Clone()}
+		if pod.GetAttributes() != nil {
+			pm[i] = &schedulingtypes.PodMetrics{Pod: pod.GetPod().Clone(), MetricsState: pod.GetMetrics().Clone(), AttributeMap: pod.GetAttributes().Clone()}
+		} else {
+			pm[i] = &schedulingtypes.PodMetrics{Pod: pod.GetPod().Clone(), MetricsState: pod.GetMetrics().Clone(), AttributeMap: datalayer.NewAttributes()}
+		}
 	}
 
 	return pm
@@ -315,6 +353,29 @@ func (d *Director) runPreRequestPlugins(ctx context.Context, request *scheduling
 	}
 }
 
+// TODO: Execute plugins in parallel once DAG execution is supported.
+// runPrepareDataPlugins executes PrepareDataPlugins sequentially.
+func (d *Director) runPrepareDataPlugins(ctx context.Context,
+	request *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) error {
+	return prepareDataPluginsWithTimeout(
+		prepareDataTimeout, d.requestControlPlugins.prepareDataPlugins, ctx, request, pods)
+
+}
+
+func (d *Director) runAdmissionPlugins(ctx context.Context,
+	request *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) bool {
+	loggerDebug := log.FromContext(ctx).V(logutil.DEBUG)
+	for _, plugin := range d.requestControlPlugins.admissionPlugins {
+		loggerDebug.Info("Running AdmitRequest plugin", "plugin", plugin.TypedName())
+		if denyReason := plugin.AdmitRequest(ctx, request, pods); denyReason != nil {
+			loggerDebug.Info("AdmitRequest plugin denied the request", "plugin", plugin.TypedName(), "reason", denyReason.Error())
+			return false
+		}
+		loggerDebug.Info("Completed running AdmitRequest plugin successfully", "plugin", plugin.TypedName())
+	}
+	return true
+}
+
 func (d *Director) runResponseReceivedPlugins(ctx context.Context, request *schedulingtypes.LLMRequest, response *Response, targetPod *backend.Pod) {
 	loggerDebug := log.FromContext(ctx).V(logutil.DEBUG)
 	for _, plugin := range d.requestControlPlugins.responseReceivedPlugins {
diff --git a/pkg/epp/requestcontrol/plugin_executor.go b/pkg/epp/requestcontrol/plugin_executor.go
new file mode 100644
index 0000000..be8d4ba
--- /dev/null
+++ b/pkg/epp/requestcontrol/plugin_executor.go
@@ -0,0 +1,51 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package requestcontrol
+
+import (
+	"context"
+	"errors"
+	"time"
+
+	schedulingtypes "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
+)
+
+// prepareDataPluginsWithTimeout executes the PrepareRequestData plugins with retries and timeout.
+func prepareDataPluginsWithTimeout(timeout time.Duration, plugins []PrepareDataPlugin,
+	ctx context.Context, request *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) error {
+	errCh := make(chan error, 1)
+	// Execute plugins sequentially in a separate goroutine
+	go func() {
+		for _, plugin := range plugins {
+			err := plugin.PrepareRequestData(ctx, request, pods)
+			if err != nil {
+				errCh <- errors.New("prepare data plugin " + plugin.TypedName().String() + " failed: " + err.Error())
+				return
+			}
+		}
+		errCh <- nil
+	}()
+
+	select {
+	case <-ctx.Done():
+		return ctx.Err()
+	case err := <-errCh:
+		return err
+	case <-time.After(timeout):
+		return errors.New("prepare data plugin timed out")
+	}
+}
diff --git a/pkg/epp/requestcontrol/plugins.go b/pkg/epp/requestcontrol/plugins.go
index 30f31f0..8c66020 100644
--- a/pkg/epp/requestcontrol/plugins.go
+++ b/pkg/epp/requestcontrol/plugins.go
@@ -57,3 +57,21 @@ type ResponseComplete interface {
 	plugins.Plugin
 	ResponseComplete(ctx context.Context, request *types.LLMRequest, response *Response, targetPod *backend.Pod)
 }
+
+// PrepareRequestData is called by the director before scheduling requests.
+// PrepareDataPlugin plugin is implemented by data producers which produce data from different sources.
+type PrepareDataPlugin interface {
+	plugins.ProducerPlugin
+	plugins.ConsumerPlugin
+	PrepareRequestData(ctx context.Context, request *types.LLMRequest, pods []types.Pod) error
+}
+
+// AdmissionPlugin is called by the director after the prepare data phase and before scheduling.
+// When a request has to go through multiple AdmissionPlugin,
+// the request is admitted only if all plugins say that the request should be admitted.
+type AdmissionPlugin interface {
+	plugins.Plugin
+	// AdmitRequest returns the denial reason, wrapped as error if the request is denied.
+	// If the request is allowed, it returns nil.
+	AdmitRequest(ctx context.Context, request *types.LLMRequest, pods []types.Pod) error
+}
diff --git a/pkg/epp/requestcontrol/request_control_config.go b/pkg/epp/requestcontrol/request_control_config.go
index ffa6c66..9701be9 100644
--- a/pkg/epp/requestcontrol/request_control_config.go
+++ b/pkg/epp/requestcontrol/request_control_config.go
@@ -23,6 +23,8 @@ import (
 // NewConfig creates a new Config object and returns its pointer.
 func NewConfig() *Config {
 	return &Config{
+		admissionPlugins:         []AdmissionPlugin{},
+		prepareDataPlugins:       []PrepareDataPlugin{},
 		preRequestPlugins:        []PreRequest{},
 		responseReceivedPlugins:  []ResponseReceived{},
 		responseStreamingPlugins: []ResponseStreaming{},
@@ -32,6 +34,8 @@ func NewConfig() *Config {
 
 // Config provides a configuration for the requestcontrol plugins.
 type Config struct {
+	admissionPlugins         []AdmissionPlugin
+	prepareDataPlugins       []PrepareDataPlugin
 	preRequestPlugins        []PreRequest
 	responseReceivedPlugins  []ResponseReceived
 	responseStreamingPlugins []ResponseStreaming
@@ -66,10 +70,21 @@ func (c *Config) WithResponseCompletePlugins(plugins ...ResponseComplete) *Confi
 	return c
 }
 
+// WithPrepareDataPlugins sets the given plugins as the PrepareData plugins.
+func (c *Config) WithPrepareDataPlugins(plugins ...PrepareDataPlugin) *Config {
+	c.prepareDataPlugins = plugins
+	return c
+}
+
+// WithAdmissionPlugins sets the given plugins as the AdmitRequest plugins.
+func (c *Config) WithAdmissionPlugins(plugins ...AdmissionPlugin) *Config {
+	c.admissionPlugins = plugins
+	return c
+}
+
 // AddPlugins adds the given plugins to the Config.
 // The type of each plugin is checked and added to the corresponding list of plugins in the Config.
 // If a plugin implements multiple plugin interfaces, it will be added to each corresponding list.
-
 func (c *Config) AddPlugins(pluginObjects ...plugins.Plugin) {
 	for _, plugin := range pluginObjects {
 		if preRequestPlugin, ok := plugin.(PreRequest); ok {
@@ -84,5 +99,8 @@ func (c *Config) AddPlugins(pluginObjects ...plugins.Plugin) {
 		if responseCompletePlugin, ok := plugin.(ResponseComplete); ok {
 			c.responseCompletePlugins = append(c.responseCompletePlugins, responseCompletePlugin)
 		}
+		if prepareDataPlugin, ok := plugin.(PrepareDataPlugin); ok {
+			c.prepareDataPlugins = append(c.prepareDataPlugins, prepareDataPlugin)
+		}
 	}
 }
diff --git a/pkg/epp/saturationdetector/config.go b/pkg/epp/saturationdetector/config.go
index b429d1e..629c3a6 100644
--- a/pkg/epp/saturationdetector/config.go
+++ b/pkg/epp/saturationdetector/config.go
@@ -16,12 +16,7 @@ limitations under the License.
 package saturationdetector
 
 import (
-	"fmt"
 	"time"
-
-	"sigs.k8s.io/controller-runtime/pkg/log"
-
-	envutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/env"
 )
 
 // Default configuration values
@@ -36,37 +31,3 @@ const (
 	// that should be fine.
 	DefaultMetricsStalenessThreshold = 200 * time.Millisecond
 )
-
-// Environment variable names for SaturationDetector configuration
-const (
-	EnvSdQueueDepthThreshold       = "SD_QUEUE_DEPTH_THRESHOLD"
-	EnvSdKVCacheUtilThreshold      = "SD_KV_CACHE_UTIL_THRESHOLD"
-	EnvSdMetricsStalenessThreshold = "SD_METRICS_STALENESS_THRESHOLD"
-)
-
-// LoadConfigFromEnv loads SaturationDetector Config from environment variables.
-func LoadConfigFromEnv() *Config {
-	// Use a default logger for initial configuration loading.
-	logger := log.Log.WithName("saturation-detector-config")
-
-	cfg := &Config{}
-
-	cfg.QueueDepthThreshold = envutil.GetEnvInt(EnvSdQueueDepthThreshold, DefaultQueueDepthThreshold, logger)
-	if cfg.QueueDepthThreshold <= 0 {
-		cfg.QueueDepthThreshold = DefaultQueueDepthThreshold
-	}
-
-	cfg.KVCacheUtilThreshold = envutil.GetEnvFloat(EnvSdKVCacheUtilThreshold, DefaultKVCacheUtilThreshold, logger)
-	if cfg.KVCacheUtilThreshold <= 0 || cfg.KVCacheUtilThreshold >= 1 {
-		cfg.KVCacheUtilThreshold = DefaultKVCacheUtilThreshold
-	}
-
-	cfg.MetricsStalenessThreshold = envutil.GetEnvDuration(EnvSdMetricsStalenessThreshold, DefaultMetricsStalenessThreshold, logger)
-	if cfg.MetricsStalenessThreshold <= 0 {
-		cfg.MetricsStalenessThreshold = DefaultMetricsStalenessThreshold
-	}
-
-	// NewDetector validates the config and assigns defaults.
-	logger.Info("SaturationDetector configuration loaded from env", "config", fmt.Sprintf("%+v", cfg))
-	return cfg
-}
diff --git a/pkg/epp/scheduling/framework/plugins/multi/prefix/OWNERS b/pkg/epp/scheduling/framework/plugins/multi/prefix/OWNERS
new file mode 100644
index 0000000..74a7d64
--- /dev/null
+++ b/pkg/epp/scheduling/framework/plugins/multi/prefix/OWNERS
@@ -0,0 +1,4 @@
+# See the OWNERS docs at https://go.k8s.io/owners
+
+approvers:
+- liu-cong
diff --git a/pkg/epp/scheduling/types/types.go b/pkg/epp/scheduling/types/types.go
index 6f9bec8..8e0553f 100644
--- a/pkg/epp/scheduling/types/types.go
+++ b/pkg/epp/scheduling/types/types.go
@@ -24,6 +24,7 @@ import (
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
 )
 
 const nilString = "<nil>"
@@ -191,6 +192,9 @@ type Pod interface {
 	GetPod() *backend.Pod
 	GetMetrics() *backendmetrics.MetricsState
 	String() string
+	Get(string) (datalayer.Cloneable, bool)
+	Put(string, datalayer.Cloneable)
+	Keys() []string
 }
 
 type ScoredPod struct {
@@ -217,6 +221,7 @@ func (pm *PodMetrics) GetMetrics() *backendmetrics.MetricsState {
 type PodMetrics struct {
 	*backend.Pod
 	*backendmetrics.MetricsState
+	datalayer.AttributeMap
 }
 
 // ProfileRunResult captures the profile run result.
@@ -229,3 +234,8 @@ type SchedulingResult struct {
 	ProfileResults     map[string]*ProfileRunResult
 	PrimaryProfileName string
 }
+
+// Cloneable types support cloning of the value.
+type Cloneable interface {
+	Clone() Cloneable
+}
diff --git a/sidecars/latencypredictorasync/OWNERS b/sidecars/latencypredictorasync/OWNERS
new file mode 100644
index 0000000..f94d386
--- /dev/null
+++ b/sidecars/latencypredictorasync/OWNERS
@@ -0,0 +1,4 @@
+# See the OWNERS docs at https://go.k8s.io/owners
+
+approvers:
+- kaushikmitr
diff --git a/site-src/_includes/epp-latest.md b/site-src/_includes/epp-latest.md
index ef08a61..e090778 100644
--- a/site-src/_includes/epp-latest.md
+++ b/site-src/_includes/epp-latest.md
@@ -30,3 +30,14 @@
       --version $IGW_CHART_VERSION \
       oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
       ```
+
+=== "NGINX Gateway Fabric"
+
+      ```bash
+      export GATEWAY_PROVIDER=none
+      helm install vllm-llama3-8b-instruct \
+      --set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
+      --set provider.name=$GATEWAY_PROVIDER \
+      --version $IGW_CHART_VERSION \
+      oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
+      ```
\ No newline at end of file
diff --git a/site-src/_includes/epp.md b/site-src/_includes/epp.md
index 73e2478..a82ca68 100644
--- a/site-src/_includes/epp.md
+++ b/site-src/_includes/epp.md
@@ -30,3 +30,14 @@
       --version $IGW_CHART_VERSION \
       oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool
       ```
+
+=== "NGINX Gateway Fabric"
+
+      ```bash
+      export GATEWAY_PROVIDER=none
+      helm install vllm-llama3-8b-instruct \
+      --set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
+      --set provider.name=$GATEWAY_PROVIDER \
+      --version $IGW_CHART_VERSION \
+      oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool
+      ```
diff --git a/site-src/guides/epp-configuration/config-text.md b/site-src/guides/epp-configuration/config-text.md
index 43a0e6c..2526091 100644
--- a/site-src/guides/epp-configuration/config-text.md
+++ b/site-src/guides/epp-configuration/config-text.md
@@ -1,23 +1,14 @@
-# Configuring Plugins via YAML
+# Configuring via YAML
 
-The set of lifecycle hooks (plugins) that are used by the Inference Gateway (IGW) is determined by how
-it is configured. The IGW is primarily configured via a configuration file.
+The Inference Gateway (IGW) can be configured via a YAML file.
 
-The YAML file can either be specified as a path to a file or in-line as a parameter. The configuration defines the set of
-plugins to be instantiated along with their parameters. Each plugin can also be given a name, enabling
-the same plugin type to be instantiated multiple times, if needed (such as when configuring multiple scheduling profiles).
+At this time the YAML file based configuration allows for:
 
-Also defined is a set of SchedulingProfiles, which determine the set of plugins to be used when scheduling a request.
-If no scheduling profile is specified, a default profile, named `default` will be added and will reference all of the
-instantiated plugins.
+1. The set of the lifecycle hooks (plugins) that are used by the IGW.
+2. The configuration of the saturation detector
+3. A set of feature gates that are used to enable experimental features.
 
-The set of plugins instantiated can include a Profile Handler, which determines which SchedulingProfiles
-will be used for a particular request. A Profile Handler must be specified, unless the configuration only
-contains one profile, in which case the `SingleProfileHandler` will be used.
-
-In addition, the set of instantiated plugins can also include a picker, which chooses the actual pod to which
-the request is scheduled after filtering and scoring. If one is not referenced in a SchedulingProfile, an
-instance of `MaxScorePicker` will be added to the SchedulingProfile in question.
+The YAML file can either be specified as a path to a file or in-line as a parameter.
 
 ***NOTE***: While the configuration text looks like a Kubernetes CRD, it is
 **NOT** a Kubernetes CRD. Specifically, the config is not reconciled upon, and is only read on startup.
@@ -33,10 +24,46 @@ plugins:
 schedulingProfiles:
 - ....
 - ....
+saturationDetector:
+  ...
+featureGates:
+  ...
 ```
 
 The first two lines of the configuration are constant and must appear as is.
 
+The plugins section defines the set of plugins that will be instantiated and their parameters. This section is described in more detail in the section [Configuring Plugins via text](#configuring-plugins-via-text)
+
+The schedulingProfiles section defines the set of scheduling profiles that can be used in scheduling
+requests to pods. This section is described in more detail in the section [Configuring Plugins via YAML](#configuring-plugins-via-yaml)
+
+The saturationDetector section configures the saturation detector, which is used to determine if special
+action needs to eb taken due to the system being overloaded or saturated. This section is described in more detail in the section [Saturation Detector configuration](#saturation-detector-configuration)
+
+The featureGates sections allows the enablement of experimental features of the IGW. This section is
+described in more detail in the section [Feature Gates](#feature-gates)
+
+## Configuring Plugins via YAML
+
+The set of plugins that are used by the IGW is determined by how it is configured. The IGW is
+primarily configured via a configuration file.
+
+The configuration defines the set of plugins to be instantiated along with their parameters.
+Each plugin can also be given a name, enabling the same plugin type to be instantiated multiple
+times, if needed (such as when configuring multiple scheduling profiles).
+
+Also defined is a set of SchedulingProfiles, which determine the set of plugins to be used when scheduling
+a request. If one is not defined, a default one names `default` will be added and will reference all of
+the instantiated plugins.
+
+The set of plugins instantiated can include a Profile Handler, which determines which SchedulingProfiles
+will be used for a particular request. A Profile Handler must be specified, unless the configuration only
+contains one profile, in which case the `SingleProfileHandler` will be used.
+
+In addition, the set of instantiated plugins can also include a picker, which chooses the actual pod to which
+the request is scheduled after filtering and scoring. If one is not referenced in a SchedulingProfile, an
+instance of `MaxScorePicker` will be added to the SchedulingProfile in question.
+
 The plugins section defines the set of plugins that will be instantiated and their parameters.
 Each entry in this section has the following form:
 
@@ -184,7 +211,7 @@ schedulingProfiles:
   -pluginRef: max-score-picker
 ```
 
-## Plugin Configuration
+### Plugin Configuration
 
 This section describes how to setup the various plugins that are available with the IGW.
 
@@ -269,3 +296,57 @@ scored higher (since it's more available to serve new request).
 
 - *Type*: lora-affinity-scorer
 - *Parameters*: none
+
+## Saturation Detector configuration
+
+The Saturation Detector is used to determine if the the cluster is overloaded, i.e. saturated. When
+the cluster is saturated special actions will be taken depending what has been enabled. At this time, sheddable requests will be dropped.
+
+The Saturation Detector determines that the cluster is saturated by looking at the following metrics provided by the inference servers:
+
+- Backed waiting queue size
+- KV cache utilization
+- Metrics staleness
+
+The Saturation Detector is configured via the saturationDetector section of the overall configuration.
+It has the following form:
+
+```yaml
+saturationDetector:
+  queueDepthThreshold: 8
+  kvCacheUtilThreshold: 0.75
+  metricsStalenessThreshold: 150ms
+```
+
+The various sub-fields of the saturationDetector section are:
+
+- The `queueDepthThreshold` field which defines the backend waiting queue size above which a
+pod is considered to have insufficient capacity for new requests. This field is optional, if
+omitted a value of `5` will be used.
+- The `kvCacheUtilThreshold` field which defines the KV cache utilization (0.0 to 1.0) above
+which a pod is considered to have insufficient capacity. This field is optional, if omitted
+a value of `0.8` will be used.
+- The `metricsStalenessThreshold` field which defines how old a pod's metrics can be. If a pod's
+metrics are older than this, it might be excluded from "good capacity" considerations or treated
+as having no capacity for safety. This field is optional, if omitted a value of `200ms` will be used.
+
+## Feature Gates
+
+The Feature Gates section allows for the enabling of experimental features of the IGW. These experimental
+features are all disabled unless you explicitly enable them one by one.
+
+The Feature Gates section has the follwoing form:
+
+```yaml
+featureGates:
+- dataLayer
+- flowControl
+```
+
+The Feature Gates section is an array of flags, each of which enables one experimental feature.
+The available values for these elements are:
+
+- `dataLayer` which, if present, enables the experimental Datalayer APIs.
+- `flowControl` which, if present, enables the experimental FlowControl feature.
+
+In all cases if the appropriate element isn't present, that experimental feature will be disabled.
\ No newline at end of file
diff --git a/site-src/guides/getting-started-latest.md b/site-src/guides/getting-started-latest.md
index 7629f4a..4db7bc1 100644
--- a/site-src/guides/getting-started-latest.md
+++ b/site-src/guides/getting-started-latest.md
@@ -199,6 +199,69 @@ kubectl apply -k https://github.com/kubernetes-sigs/gateway-api-inference-extens
          kubectl get httproute llm-route -o yaml
          ```
 
+=== "NGINX Gateway Fabric"
+
+      NGINX Gateway Fabric is an implementation of the Gateway API that supports the Inference Extension. Follow these steps to deploy an Inference Gateway using NGINX Gateway Fabric.
+
+      1. Requirements
+
+         - Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed (Standard or Experimental channel).
+         - [Helm](https://helm.sh/docs/intro/install/) installed.
+         - A Kubernetes cluster with LoadBalancer or NodePort access.
+
+      2. Install NGINX Gateway Fabric with the Inference Extension enabled by setting the `nginxGateway.gwAPIInferenceExtension.enable=true` Helm value
+
+         ```bash 
+         helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway --set nginxGateway.gwAPIInferenceExtension.enable=true
+         ```
+         This enables NGINX Gateway Fabric to watch and manage Inference Extension resources such as InferencePool and InferenceObjective.
+
+      3. Deploy the Gateway
+
+         ```bash
+         kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/nginxgatewayfabric/gateway.yaml
+         ```
+
+      4. Verify the Gateway status
+         
+         Ensure that the Gateway is running and has been assigned an address:
+
+         ```bash
+         kubectl get gateway inference-gateway
+         ```
+
+         Check that the Gateway has been successfully provisioned and that its status shows Programmed=True
+      
+      5. Deploy the HTTPRoute
+         
+         Create the HTTPRoute resource to route traffic to your InferencePool:
+
+         ```bash
+         kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/nginxgatewayfabric/httproute.yaml
+         ```
+
+      6. Verify the route status
+
+         Check that the HTTPRoute was successfully configured and references were resolved:
+
+         ```bash
+         kubectl get httproute llm-route -o yaml
+         ```
+         
+         The route status should include Accepted=True and ResolvedRefs=True.
+
+      7. Verify the InferencePool Status
+
+         Make sure the InferencePool is active before sending traffic.
+
+         ```bash
+         kubectl describe inferencepools.inference.networking.k8s.io vllm-llama3-8b-instruct
+         ```
+
+         Check that the status shows Accepted=True and ResolvedRefs=True. This confirms the InferencePool is ready to handle traffic.
+      
+       For more information, see the [NGINX Gateway Fabric - Inference Gateway Setup guide](https://docs.nginx.com/nginx-gateway-fabric/how-to/gateway-api-inference-extension/#overview)
+
 ### Deploy InferenceObjective (Optional)
 
 Deploy the sample InferenceObjective which allows you to specify priority of requests.
@@ -290,3 +353,27 @@ Deploy the sample InferenceObjective which allows you to specify priority of req
          ```bash
          kubectl delete ns kgateway-system
          ```
+
+=== "NGINX Gateway Fabric"
+
+      Follow these steps to remove the NGINX Gateway Fabric Inference Gateway and all related resources.
+
+
+      1. Remove Inference Gateway and HTTPRoute:
+
+         ```bash
+         kubectl delete -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/nginxgatewayfabric/gateway.yaml --ignore-not-found
+         kubectl delete -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/nginxgatewayfabric/httproute.yaml --ignore-not-found
+         ```
+
+      2. Uninstall NGINX Gateway Fabric:
+
+         ```bash
+         helm uninstall ngf -n nginx-gateway
+         ```
+
+      3. Clean up namespace:
+   
+         ```bash
+         kubectl delete ns nginx-gateway
+         ```
\ No newline at end of file
diff --git a/site-src/guides/index.md b/site-src/guides/index.md
index db47db0..b655e56 100644
--- a/site-src/guides/index.md
+++ b/site-src/guides/index.md
@@ -88,6 +88,22 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
          helm upgrade -i --namespace kgateway-system --version $KGTW_VERSION kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --set inferenceExtension.enabled=true
          ```
 
+=== "NGINX Gateway Fabric"
+
+      1. Requirements
+
+         - Gateway API [CRDs](https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) installed (Standard or Experimental channel).
+         - [Helm](https://helm.sh/docs/intro/install/) installed.
+         - A Kubernetes cluster with LoadBalancer or NodePort access.
+
+      2. Install NGINX Gateway Fabric with the Inference Extension enabled by setting the `nginxGateway.gwAPIInferenceExtension.enable=true` Helm value
+
+         ```bash 
+         helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway --set nginxGateway.gwAPIInferenceExtension.enable=true
+         ```
+         This enables NGINX Gateway Fabric to watch and manage Inference Extension resources such as InferencePool and InferenceObjective.
+
+
 ### Deploy the InferencePool and Endpoint Picker Extension
 
    Install an InferencePool named `vllm-llama3-8b-instruct` that selects from endpoints with label `app: vllm-llama3-8b-instruct` and listening on port 8000. The Helm install command automatically installs the endpoint-picker, InferencePool along with provider specific resources.
@@ -195,6 +211,57 @@ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extens
          kubectl get httproute llm-route -o yaml
          ```
 
+=== "NGINX Gateway Fabric"
+
+      NGINX Gateway Fabric is an implementation of the Gateway API that supports the Inference Extension. Follow these steps to deploy an Inference Gateway using NGINX Gateway Fabric.
+
+      1. Deploy the Gateway
+
+         ```bash
+         kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/nginxgatewayfabric/gateway.yaml
+         ```
+
+      2. Verify the Gateway status
+         
+         Ensure that the Gateway is running and has been assigned an address:
+
+         ```bash
+         kubectl get gateway inference-gateway
+         ```
+
+         Check that the Gateway has been successfully provisioned and that its status shows Programmed=True
+      
+      3. Deploy the HTTPRoute
+         
+         Create the HTTPRoute resource to route traffic to your InferencePool:
+
+         ```bash
+         kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/nginxgatewayfabric/httproute.yaml
+         ```
+
+      4. Verify the route status
+
+         Check that the HTTPRoute was successfully configured and references were resolved:
+
+         ```bash
+         kubectl get httproute llm-route -o yaml
+         ```
+         
+         The route status should include Accepted=True and ResolvedRefs=True.
+
+      5. Verify the InferencePool Status
+
+         Make sure the InferencePool is active before sending traffic.
+
+         ```bash
+         kubectl describe inferencepools.inference.networking.k8s.io vllm-llama3-8b-instruct
+         ```
+
+         Check that the status shows Accepted=True and ResolvedRefs=True. This confirms the InferencePool is ready to handle traffic.
+      
+       For more information, see the [NGINX Gateway Fabric - Inference Gateway Setup guide](https://docs.nginx.com/nginx-gateway-fabric/how-to/gateway-api-inference-extension/#overview)
+
+
 ### Deploy InferenceObjective (Optional)
 
 Deploy the sample InferenceObjective which allows you to specify priority of requests.
@@ -287,3 +354,27 @@ Deploy the sample InferenceObjective which allows you to specify priority of req
          ```bash
          kubectl delete ns kgateway-system
          ```
+
+=== "NGINX Gateway Fabric"
+
+      Follow these steps to remove the NGINX Gateway Fabric Inference Gateway and all related resources.
+
+
+      1. Remove Inference Gateway and HTTPRoute:
+
+         ```bash
+         kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/nginxgatewayfabric/gateway.yaml --ignore-not-found
+         kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.0.2/config/manifests/gateway/nginxgatewayfabric/httproute.yaml --ignore-not-found
+         ```
+
+      2. Uninstall NGINX Gateway Fabric:
+
+         ```bash
+         helm uninstall ngf -n nginx-gateway
+         ```
+
+      3. Clean up namespace:
+   
+         ```bash
+         kubectl delete ns nginx-gateway
+         ```
diff --git a/site-src/guides/troubleshooting.md b/site-src/guides/troubleshooting.md
index 1f08619..dc95e95 100644
--- a/site-src/guides/troubleshooting.md
+++ b/site-src/guides/troubleshooting.md
@@ -23,9 +23,17 @@ This error indicates that the entire inference pool has exceeded its saturation
     * **v1.0.0 and later**: Ensure the `InferenceObjective` you're using has a `priority` greater than or equal to 0. A negative priority can cause requests to be dropped.
 
 * Pool Thresholds: Check the defined pool [thresholds](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/f36111cab0ed5a309d1eafade896d4f37ab623a6/pkg/epp/saturationdetector/config.go#L41) to understand the saturation limits. Currently, we use three main metrics to assess the system's load:
-    * `DefaultQueueDepthThreshold`: This is the maximum number of requests waiting in the queue for a backend. The default value is 5. If the queue for a model server exceeds this number, the saturation detector may consider the system under pressure. To override this, set the `SD_QUEUE_DEPTH_THRESHOLD` environment variable.
-    * `DefaultKVCacheUtilThreshold`: This is the maximum utilization of the Key-Value (KV) cache on the model server, expressed as a decimal from 0.0 to 1.0. The default is 0.8, or 80%. The KV cache stores attention keys and values to speed up inference for subsequent tokens. When its utilization exceeds this threshold, it's an indication that the model server is nearing its memory capacity and may be becoming saturated. To override this, set the `SD_KV_CACHE_UTIL_THRESHOLD` environment variable.
-    * `DefaultMetricsStalenessThreshold`: This defines the maximum age of metrics data before it's considered outdated. The default is 200 milliseconds. The saturation detector needs up-to-date metrics to make accurate decisions about system load. If the metrics are older than this threshold, the detector won't use them. This value is tied to how often metrics are refreshed, and setting it slightly higher ensures that there's always fresh data available. To override this, set the `SD_METRICS_STALENESS_THRESHOLD` environment variable.
+    * `DefaultQueueDepthThreshold`: This is the maximum number of requests waiting in the queue for a backend. The default value is 5. If the queue for a model server exceeds this number, the saturation detector may consider the system under pressure. 
+    To override this, set the `queueDepthThreshold` field in the `saturationDetector` section of the text based configuration.
+    See [Saturation Detector configuration](../epp-configuration/config-text#saturation-detector-configuration).
+    <br>**Note:** The use of the `SD_QUEUE_DEPTH_THRESHOLD` environment variable to override this is now deprecated.
+    * `DefaultKVCacheUtilThreshold`: This is the maximum utilization of the Key-Value (KV) cache on the model server, expressed as a decimal from 0.0 to 1.0. The default is 0.8, or 80%. The KV cache stores attention keys and values to speed up inference for subsequent tokens. When its utilization exceeds this threshold, it's an indication that the model server is nearing its memory capacity and may be becoming saturated.
+    To override this, set the `kvCacheUtilThreshold` field in the `saturationDetector` section of the text based configuration.
+    See [Saturation Detector configuration](../epp-configuration/config-text#saturation-detector-configuration).
+    <br>**Note:** The use of the `SD_KV_CACHE_UTIL_THRESHOLD` environment variable to override this is now deprecated.
+    * `DefaultMetricsStalenessThreshold`: This defines the maximum age of metrics data before it's considered outdated. The default is 200 milliseconds. The saturation detector needs up-to-date metrics to make accurate decisions about system load. If the metrics are older than this threshold, the detector won't use them. This value is tied to how often metrics are refreshed, and setting it slightly higher ensures that there's always fresh data available.
+    To override this, set the `metricsStalenessThreshold` field in the `saturationDetector` section of the text based configuration. See [Saturation Detector configuration](../epp-configuration/config-text#saturation-detector-configuration).
+    <br>**Note:** The use of the `SD_METRICS_STALENESS_THRESHOLD` environment variable to override this is now deprecated.
 
 ## 500 Internal Server Error
 ### `fault filter abort`
diff --git a/site-src/implementations/gateways.md b/site-src/implementations/gateways.md
index 8c7ee8d..7ea67f1 100644
--- a/site-src/implementations/gateways.md
+++ b/site-src/implementations/gateways.md
@@ -9,6 +9,7 @@ This project has several implementations that are planned or in progress:
   - [Istio](#istio)
   - [Kgateway](#kgateway)
   - [Kubvernor](#kubvernor)
+  - [NGINX Gateway Fabric](#nginx-gateway-fabric)
 
 [1]:#alibaba-cloud-container-service-for-kubernetes
 [2]:#envoy-ai-gateway
@@ -16,6 +17,7 @@ This project has several implementations that are planned or in progress:
 [4]:#istio
 [5]:#kgateway
 [6]:#kubvernor
+[7]:#nginx-gateway-fabric
 
 Agentgateway can run independently or can be managed by [Kgateway](https://kgateway.dev/).
 
@@ -98,3 +100,10 @@ Kgateway supports Inference Gateway with the [agentgateway](https://agentgateway
 [krg]:https://github.com/kubvernor/kubvernor
 [krgu]: https://github.com/kubvernor/kubvernor/blob/main/README.md
 
+## NGINX Gateway Fabric
+
+[NGINX Gateway Fabric][nginx-gateway-fabric] is an open-source project that provides an implementation of the Gateway API using [NGINX][nginx] as the data plane. The goal of this project is to implement the core Gateway API to configure an HTTP or TCP/UDP load balancer, reverse-proxy, or API gateway for applications running on Kubernetes. You can find the comprehensive NGINX Gateway Fabric user documentation on the [NGINX Documentation][nginx-docs] website.
+
+[nginx-gateway-fabric]: https://github.com/nginx/nginx-gateway-fabric
+[nginx]:https://nginx.org/
+[nginx-docs]:https://docs.nginx.com/nginx-gateway-fabric/
\ No newline at end of file


Test patch:
diff --git a/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewritestatus.go b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewritestatus.go
new file mode 100644
index 0000000..8004c53
--- /dev/null
+++ b/client-go/applyconfiguration/apix/v1alpha2/inferencemodelrewritestatus.go
@@ -0,0 +1,48 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	v1 "k8s.io/client-go/applyconfigurations/meta/v1"
+)
+
+// InferenceModelRewriteStatusApplyConfiguration represents a declarative configuration of the InferenceModelRewriteStatus type for use
+// with apply.
+type InferenceModelRewriteStatusApplyConfiguration struct {
+	Conditions []v1.ConditionApplyConfiguration `json:"conditions,omitempty"`
+}
+
+// InferenceModelRewriteStatusApplyConfiguration constructs a declarative configuration of the InferenceModelRewriteStatus type for use with
+// apply.
+func InferenceModelRewriteStatus() *InferenceModelRewriteStatusApplyConfiguration {
+	return &InferenceModelRewriteStatusApplyConfiguration{}
+}
+
+// WithConditions adds the given value to the Conditions field in the declarative configuration
+// and returns the receiver, so that objects can be build by chaining "With" function invocations.
+// If called multiple times, values provided by each call will be appended to the Conditions field.
+func (b *InferenceModelRewriteStatusApplyConfiguration) WithConditions(values ...*v1.ConditionApplyConfiguration) *InferenceModelRewriteStatusApplyConfiguration {
+	for i := range values {
+		if values[i] == nil {
+			panic("nil value passed to WithConditions")
+		}
+		b.Conditions = append(b.Conditions, *values[i])
+	}
+	return b
+}
diff --git a/conformance/tests/gateway_following_epp_routing_dp.go b/conformance/tests/gateway_following_epp_routing_dp.go
new file mode 100644
index 0000000..0067095
--- /dev/null
+++ b/conformance/tests/gateway_following_epp_routing_dp.go
@@ -0,0 +1,309 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tests
+
+import (
+	"errors"
+	"fmt"
+	"net/http"
+	"slices"
+	"strconv"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"golang.org/x/sync/errgroup"
+	corev1 "k8s.io/api/core/v1"
+	"k8s.io/apimachinery/pkg/types"
+	gwhttp "sigs.k8s.io/gateway-api/conformance/utils/http"
+	"sigs.k8s.io/gateway-api/conformance/utils/suite"
+	"sigs.k8s.io/gateway-api/pkg/features"
+
+	"sigs.k8s.io/gateway-api-inference-extension/conformance/resources"
+	k8sutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/test"
+)
+
+// dpPorts are the data parallel ports exposed by the backend deployment. Update if the ports
+// change in conformance/resources/base.yaml.
+var dpPorts = portSet{"3000": {}, "3002": {}, "3004": {}}
+
+func init() {
+	ConformanceTests = append(ConformanceTests, GatewayFollowingEPPRoutingWithDataParallelism)
+}
+
+// GatewayFollowingEPPRoutingWithDataParallelism verifies that with multiple targetPorts (ranks)
+// the gateway still routes only to pods returned by EPP.
+var GatewayFollowingEPPRoutingWithDataParallelism = suite.ConformanceTest{
+	ShortName:   "GatewayFollowingEPPRoutingWithDataParallelism",
+	Description: "Inference gateway should restrict traffic to EPP-selected pods while EPP balances across multiple targetPorts (DP ranks)",
+	Manifests:   []string{"tests/gateway_following_epp_routing_dp.yaml"},
+	Features: []features.FeatureName{
+		features.FeatureName("SupportInferencePool"),
+		features.SupportGateway,
+	},
+	Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
+		const (
+			hostname            = "primary.example.com"
+			path                = "/primary-gateway-dp-test"
+			appPodBackendPrefix = "dp-inference-model-server" // Must match the app label in the backend pods
+		)
+
+		httpRouteNN := types.NamespacedName{Name: "httproute-for-primary-gw-dp", Namespace: resources.AppBackendNamespace}
+		gatewayNN := resources.PrimaryGatewayNN
+		poolNN := types.NamespacedName{Name: "dp-inference-pool", Namespace: resources.AppBackendNamespace}
+		backendPodLabels := map[string]string{"app": "dp-inference-model-server"}
+
+		t.Log("Verifying HTTPRoute and InferencePool are accepted and the Gateway has an address.")
+		k8sutils.HTTPRouteMustBeAcceptedAndResolved(t, s.Client, s.TimeoutConfig, httpRouteNN, gatewayNN)
+		k8sutils.InferencePoolMustBeAcceptedByParent(t, s.Client, poolNN, gatewayNN)
+		gwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, gatewayNN)
+
+		t.Logf("Fetching backend pods with labels: %v", backendPodLabels)
+		pods, err := k8sutils.GetPodsWithLabel(t, s.Client, resources.AppBackendNamespace, backendPodLabels, s.TimeoutConfig)
+		require.NoError(t, err, "Failed to get backend pods")
+		require.Len(t, pods, 3, "Expected to find 3 backend pods, found %d", len(pods))
+
+		backends := toEndpoints(pods)
+		require.Len(t, backends, 3)
+
+		podNameToIP := make(map[string]string, len(backends))
+		for _, be := range backends {
+			podNameToIP[be.Name] = be.IP
+		}
+
+		requestBody := `{
+			"model": "conformance-fake-model",
+			"prompt": "Write as if you were a critic: San Francisco"
+		}`
+
+		// Single-pod pin to ensure header filter works before main test cases.
+		for _, backend := range backends {
+			gwhttp.MakeRequestAndExpectEventuallyConsistentResponse(
+				t,
+				s.RoundTripper,
+				s.TimeoutConfig,
+				gwAddr,
+				gwhttp.ExpectedResponse{
+					Request: gwhttp.Request{
+						Host:   hostname,
+						Path:   path,
+						Method: http.MethodPost,
+						Body:   requestBody,
+						Headers: map[string]string{
+							test.HeaderTestEppEndPointSelectionKey: backend.IP,
+						},
+					},
+					Response:  gwhttp.Response{StatusCodes: []int{http.StatusOK}},
+					Backend:   backend.Name,
+					Namespace: resources.AppBackendNamespace,
+				},
+			)
+		}
+
+		testCases := []struct {
+			name                                  string
+			ipToAllowedPorts                      map[string]portSet
+			expectAllRequestsRoutedWithinPodNames []string
+		}{
+			{
+				name: "DP routes only to one designated pod (any rank)",
+				ipToAllowedPorts: map[string]portSet{
+					backends[1].IP: {}, // any of the DP ports for this IP
+				},
+				expectAllRequestsRoutedWithinPodNames: []string{backends[1].Name},
+			},
+			{
+				name: "DP routes only to two designated pods; one has a fixed rank",
+				ipToAllowedPorts: map[string]portSet{
+					backends[0].IP: {},           // any port
+					backends[2].IP: {"3002": {}}, // must be port 3002 for this IP
+				},
+				expectAllRequestsRoutedWithinPodNames: []string{backends[0].Name, backends[2].Name},
+			},
+			{
+				name: "DP routes to all pods; one pod restricted to 3000,3004",
+				ipToAllowedPorts: map[string]portSet{
+					backends[0].IP: {},
+					backends[1].IP: {"3000": {}, "3004": {}},
+					backends[2].IP: {},
+				},
+				expectAllRequestsRoutedWithinPodNames: []string{backends[0].Name, backends[1].Name, backends[2].Name},
+			},
+		}
+
+		for _, tc := range testCases {
+			t.Run(tc.name, func(t *testing.T) {
+				// Build the EPP header from the endpoint map (stable order).
+				eppHeaderValue := buildEPPHeader(tc.ipToAllowedPorts)
+
+				headers := map[string]string{
+					test.HeaderTestEppEndPointSelectionKey: eppHeaderValue,
+				}
+
+				assertTrafficOnlyReachesToExpectedPodsDP(
+					t, s, gwAddr,
+					gwhttp.ExpectedResponse{
+						Request: gwhttp.Request{
+							Host:    hostname,
+							Path:    path,
+							Method:  http.MethodPost,
+							Body:    requestBody,
+							Headers: headers,
+						},
+						Response:  gwhttp.Response{StatusCode: http.StatusOK},
+						Backend:   appPodBackendPrefix,
+						Namespace: resources.AppBackendNamespace,
+					},
+					tc.expectAllRequestsRoutedWithinPodNames,
+					podNameToIP,
+					tc.ipToAllowedPorts,
+				)
+			})
+		}
+	},
+}
+
+func assertTrafficOnlyReachesToExpectedPodsDP(
+	t *testing.T,
+	suite *suite.ConformanceTestSuite,
+	gwAddr string,
+	expected gwhttp.ExpectedResponse,
+	expectedPodNames []string,
+	podNameToIP map[string]string,
+	ipToAllowedPorts map[string]portSet,
+) {
+	t.Helper()
+	const (
+		concurrency   = 10
+		totalRequests = 100
+	)
+
+	var (
+		rt = suite.RoundTripper
+		g  errgroup.Group
+		r  = gwhttp.MakeRequest(t, &expected, gwAddr, "HTTP", "http")
+	)
+	g.SetLimit(concurrency)
+
+	for i := 0; i < totalRequests; i++ {
+		g.Go(func() error {
+			cReq, cRes, err := rt.CaptureRoundTrip(r)
+			if err != nil {
+				return fmt.Errorf("failed roundtrip: %w", err)
+			}
+			// Baseline response checks (status/namespace/backend, etc.)
+			if err := gwhttp.CompareRoundTrip(t, &r, cReq, cRes, expected); err != nil {
+				return fmt.Errorf("expectation failed: %w", err)
+			}
+			// Enforce no leakage to non-selected pods.
+			if !slices.Contains(expectedPodNames, cReq.Pod) {
+				return fmt.Errorf("unexpected pod %q (expected one of %v)", cReq.Pod, expectedPodNames)
+			}
+
+			// Validate httpPort from JSON response body vs EPP intent.
+			if cReq.HTTPPort == "" {
+				return errors.New("missing httpPort in echo JSON body response")
+			}
+			ip := podNameToIP[cReq.Pod]
+			allowed, ok := ipToAllowedPorts[ip]
+			if !ok {
+				return fmt.Errorf("pod %q (IP %s) not present in EPP selection", cReq.Pod, ip)
+			}
+			if len(allowed) > 0 {
+				if _, ok := allowed[cReq.HTTPPort]; !ok {
+					return fmt.Errorf("unexpected httpPort %q for IP %s (allowed: %v)", cReq.HTTPPort, ip, keys(allowed))
+				}
+			} else {
+				if _, ok := dpPorts[cReq.HTTPPort]; !ok {
+					return fmt.Errorf("unexpected httpPort %q for IP %s (expected one of ports %v)", cReq.HTTPPort, ip, keys(dpPorts))
+				}
+			}
+
+			return nil
+		})
+	}
+	if err := g.Wait(); err != nil {
+		t.Fatalf("Requests were not confined to expected pods or failed port checks: %v", err)
+	}
+	t.Logf("Traffic restricted to %v and httpPort validated against EPP selection", expectedPodNames)
+}
+
+type portSet map[string]struct{}
+
+type PodEndpoint struct {
+	Name  string
+	IP    string
+	Ports []string
+}
+
+func keys(m portSet) []string {
+	out := make([]string, 0, len(m))
+	for k := range m {
+		out = append(out, k)
+	}
+	return out
+}
+
+// buildEPPHeader builds the test EPP header from ip->ports. AN empty portSet => emit just "IP".
+// A non-empty => emit "IP:port" for each port. Sorted for determinism.
+func buildEPPHeader(ipToPorts map[string]portSet) string {
+	ips := make([]string, 0, len(ipToPorts))
+	for ip := range ipToPorts {
+		ips = append(ips, ip)
+	}
+	slices.Sort(ips)
+
+	var tokens []string
+	for _, ip := range ips {
+		ports := ipToPorts[ip]
+		if len(ports) == 0 {
+			tokens = append(tokens, ip)
+			continue
+		}
+		ps := keys(ports)
+		slices.Sort(ps)
+		for _, p := range ps {
+			tokens = append(tokens, fmt.Sprintf("%s:%s", ip, p))
+		}
+	}
+	return strings.Join(tokens, ",")
+}
+
+// toEndpoints extracts name, IP, and unique containerPort values per pod.
+func toEndpoints(pods []corev1.Pod) []PodEndpoint {
+	out := make([]PodEndpoint, 0, len(pods))
+	for _, p := range pods {
+		seen := map[int32]struct{}{}
+		var ports []string
+		for _, c := range p.Spec.Containers {
+			for _, cp := range c.Ports {
+				if _, ok := seen[cp.ContainerPort]; ok {
+					continue
+				}
+				seen[cp.ContainerPort] = struct{}{}
+				ports = append(ports, strconv.Itoa(int(cp.ContainerPort)))
+			}
+		}
+		out = append(out, PodEndpoint{
+			Name:  p.Name,
+			IP:    p.Status.PodIP,
+			Ports: ports,
+		})
+	}
+	return out
+}
diff --git a/conformance/tests/gateway_following_epp_routing_dp.yaml b/conformance/tests/gateway_following_epp_routing_dp.yaml
new file mode 100644
index 0000000..02f849c
--- /dev/null
+++ b/conformance/tests/gateway_following_epp_routing_dp.yaml
@@ -0,0 +1,23 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: httproute-for-primary-gw-dp
+  namespace: inference-conformance-app-backend
+spec:
+  parentRefs:
+  - group: gateway.networking.k8s.io
+    kind: Gateway
+    name: conformance-primary
+    namespace: inference-conformance-infra
+    sectionName: http
+  hostnames:
+  - "primary.example.com"
+  rules:
+  - backendRefs:
+    - group: inference.networking.k8s.io
+      kind: InferencePool
+      name: dp-inference-pool
+    matches:
+    - path:
+        type: PathPrefix
+        value: /primary-gateway-dp-test
diff --git a/pkg/epp/config/loader/configloader_test.go b/pkg/epp/config/loader/configloader_test.go
index 9e428c2..a23f223 100644
--- a/pkg/epp/config/loader/configloader_test.go
+++ b/pkg/epp/config/loader/configloader_test.go
@@ -21,6 +21,7 @@ import (
 	"encoding/json"
 	"os"
 	"testing"
+	"time"
 
 	"github.com/google/go-cmp/cmp"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -28,7 +29,9 @@ import (
 
 	configapi "sigs.k8s.io/gateway-api-inference-extension/apix/config/v1alpha1"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/config"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/multi/prefix"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
@@ -97,6 +100,10 @@ func TestLoadRawConfiguration(t *testing.T) {
 				},
 			},
 		},
+		FeatureGates: configapi.FeatureGates{datalayer.FeatureGate},
+		SaturationDetector: &configapi.SaturationDetector{
+			MetricsStalenessThreshold: metav1.Duration{Duration: 150 * time.Millisecond},
+		},
 	}
 
 	goodConfigNoProfiles := &configapi.EndpointPickerConfig{
@@ -199,6 +206,12 @@ func TestLoadRawConfigurationWithDefaults(t *testing.T) {
 				},
 			},
 		},
+		FeatureGates: configapi.FeatureGates{datalayer.FeatureGate},
+		SaturationDetector: &configapi.SaturationDetector{
+			QueueDepthThreshold:       saturationdetector.DefaultQueueDepthThreshold,
+			KVCacheUtilThreshold:      saturationdetector.DefaultKVCacheUtilThreshold,
+			MetricsStalenessThreshold: metav1.Duration{Duration: 150 * time.Millisecond},
+		},
 	}
 
 	goodConfigNoProfiles := &configapi.EndpointPickerConfig{
@@ -234,6 +247,12 @@ func TestLoadRawConfigurationWithDefaults(t *testing.T) {
 				},
 			},
 		},
+		FeatureGates: configapi.FeatureGates{},
+		SaturationDetector: &configapi.SaturationDetector{
+			QueueDepthThreshold:       saturationdetector.DefaultQueueDepthThreshold,
+			KVCacheUtilThreshold:      saturationdetector.DefaultKVCacheUtilThreshold,
+			MetricsStalenessThreshold: metav1.Duration{Duration: saturationdetector.DefaultMetricsStalenessThreshold},
+		},
 	}
 
 	tests := []testStruct{
@@ -346,10 +365,16 @@ func checkError(t *testing.T, function string, test testStruct, err error) {
 }
 
 func TestInstantiatePlugins(t *testing.T) {
+	registerNeededFeatureGates()
+	logger := logging.NewTestLogger()
+	rawConfig, _, err := LoadConfigPhaseOne([]byte(successConfigText), logger)
+	if err != nil {
+		t.Fatalf("LoadConfigPhaseOne returned unexpected error - %v", err)
+	}
 	handle := utils.NewTestHandle(context.Background())
-	_, err := LoadConfig([]byte(successConfigText), handle, logging.NewTestLogger())
+	_, err = LoadConfigPhaseTwo(rawConfig, handle, logger)
 	if err != nil {
-		t.Fatalf("LoadConfig returned unexpected error - %v", err)
+		t.Fatalf("LoadConfigPhaseTwo returned unexpected error - %v", err)
 	}
 	if len(handle.GetAllPlugins()) == 0 {
 		t.Fatalf("unexpected empty set of loaded plugins")
@@ -360,10 +385,14 @@ func TestInstantiatePlugins(t *testing.T) {
 		t.Fatalf("loaded plugins returned test1 has the wrong type %#v", t1)
 	}
 
+	rawConfig, _, err = LoadConfigPhaseOne([]byte(errorBadPluginReferenceParametersText), logger)
+	if err != nil {
+		t.Fatalf("LoadConfigPhaseTwo returned unexpected error - %v", err)
+	}
 	handle = utils.NewTestHandle(context.Background())
-	_, err = LoadConfig([]byte(errorBadPluginReferenceParametersText), handle, logging.NewTestLogger())
+	_, err = LoadConfigPhaseTwo(rawConfig, handle, logger)
 	if err == nil {
-		t.Fatalf("LoadConfig did not return error as expected ")
+		t.Fatalf("LoadConfigPhaseTwo did not return error as expected ")
 	}
 }
 
@@ -425,25 +454,43 @@ func TestLoadConfig(t *testing.T) {
 			configText: errorMultiProfilesUseSingleProfileHandlerText,
 			wantErr:    true,
 		},
+		{
+			name:       "errorUnknownFeatureGate",
+			configText: errorUnknownFeatureGateText,
+			wantErr:    true,
+		},
 	}
 
+	registerNeededFeatureGates()
 	registerNeededPlgugins()
 
 	logger := logging.NewTestLogger()
 	for _, test := range tests {
-		handle := utils.NewTestHandle(context.Background())
-		_, err := LoadConfig([]byte(test.configText), handle, logger)
+		rawConfig, _, err := LoadConfigPhaseOne([]byte(test.configText), logger)
 		if err != nil {
 			if !test.wantErr {
-				t.Errorf("LoadConfig returned an unexpected error. error %v", err)
+				t.Errorf("LoadConfigPhaseOne returned an unexpected error. error %v", err)
 			}
 			t.Logf("error was %s", err)
 		} else if test.wantErr {
-			t.Errorf("LoadConfig did not return an expected error (%s)", test.name)
+			handle := utils.NewTestHandle(context.Background())
+			_, err = LoadConfigPhaseTwo(rawConfig, handle, logger)
+			if err != nil {
+				if !test.wantErr {
+					t.Errorf("LoadConfigPhaseOne returned an unexpected error. error %v", err)
+				}
+				t.Logf("error was %s", err)
+			} else if test.wantErr {
+				t.Errorf("LoadConfig did not return an expected error (%s)", test.name)
+			}
 		}
 	}
 }
 
+func registerNeededFeatureGates() {
+	RegisterFeatureGate(datalayer.FeatureGate)
+}
+
 func registerNeededPlgugins() {
 	plugins.Register(prefix.PrefixCachePluginType, prefix.PrefixCachePluginFactory)
 	plugins.Register(picker.MaxScorePickerType, picker.MaxScorePickerFactory)
@@ -452,6 +499,64 @@ func registerNeededPlgugins() {
 	plugins.Register(profile.SingleProfileHandlerType, profile.SingleProfileHandlerFactory)
 }
 
+func TestNewDetector(t *testing.T) {
+	tests := []struct {
+		name           string
+		config         *configapi.SaturationDetector
+		expectedConfig saturationdetector.Config
+	}{
+		{
+			name: "Valid config",
+			config: &configapi.SaturationDetector{
+				QueueDepthThreshold:       10,
+				KVCacheUtilThreshold:      0.8,
+				MetricsStalenessThreshold: metav1.Duration{Duration: 100 * time.Millisecond},
+			},
+			expectedConfig: saturationdetector.Config{
+				QueueDepthThreshold:       10,
+				KVCacheUtilThreshold:      0.8,
+				MetricsStalenessThreshold: 100 * time.Millisecond,
+			},
+		},
+		{
+			name: "invalid thresholds, fallback to default",
+			config: &configapi.SaturationDetector{
+				QueueDepthThreshold:       -1,
+				KVCacheUtilThreshold:      -5.0,
+				MetricsStalenessThreshold: metav1.Duration{Duration: 0 * time.Second},
+			},
+			expectedConfig: saturationdetector.Config{
+				QueueDepthThreshold:       saturationdetector.DefaultQueueDepthThreshold,
+				KVCacheUtilThreshold:      saturationdetector.DefaultKVCacheUtilThreshold,
+				MetricsStalenessThreshold: saturationdetector.DefaultMetricsStalenessThreshold,
+			},
+		},
+		{
+			name: "kv cache threshold above range, fallback to default",
+			config: &configapi.SaturationDetector{
+				QueueDepthThreshold:       10,
+				KVCacheUtilThreshold:      1.5,
+				MetricsStalenessThreshold: metav1.Duration{Duration: 100 * time.Millisecond},
+			},
+			expectedConfig: saturationdetector.Config{
+				QueueDepthThreshold:       10,
+				KVCacheUtilThreshold:      saturationdetector.DefaultKVCacheUtilThreshold,
+				MetricsStalenessThreshold: 100 * time.Millisecond,
+			},
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			// validate configuration values are loaded from configuration struct properly, including the use of default values when provided value is invalid.
+			sdConfig := loadSaturationDetectorConfig(test.config)
+			if diff := cmp.Diff(test.expectedConfig, *sdConfig); diff != "" {
+				t.Errorf("Unexpected output (-want +got): %v", diff)
+			}
+		})
+	}
+}
+
 // The following multi-line string constants, cause false positive lint errors (dupword)
 
 // valid configuration
@@ -479,6 +584,10 @@ schedulingProfiles:
   - pluginRef: test-two
     weight: 50
   - pluginRef: testPicker
+featureGates:
+- dataLayer
+saturationDetector:
+  metricsStalenessThreshold: 150ms
 `
 
 // success with missing scheduling profiles
@@ -644,6 +753,21 @@ schedulingProfiles:
   - pluginRef: test2
 `
 
+// error with an unknown feature gate
+//
+//nolint:dupword
+const errorUnknownFeatureGateText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: test1
+  type: test-one
+  parameters:
+    threshold: 10
+featureGates:
+- qwerty
+`
+
 // compile-time type validation
 var _ framework.Filter = &test1{}
 
@@ -783,6 +907,8 @@ schedulingProfiles:
   - pluginRef: prefixCacheScorer
     weight: 50
   - pluginRef: maxScorePicker
+featureGates:
+- dataLayer
 `
 
 // valid configuration, with default weight for scorer
diff --git a/pkg/epp/datalayer/attributemap_test.go b/pkg/epp/datalayer/attributemap_test.go
index a6697bd..c4ad569 100644
--- a/pkg/epp/datalayer/attributemap_test.go
+++ b/pkg/epp/datalayer/attributemap_test.go
@@ -43,6 +43,9 @@ func TestExpectPutThenGetToMatch(t *testing.T) {
 	dv, ok := got.(*dummy)
 	assert.True(t, ok, "expected value to be of type *dummy")
 	assert.Equal(t, "foo", dv.Text)
+
+	_, ok = attrs.Get("b")
+	assert.False(t, ok, "expected key not to exist")
 }
 
 func TestExpectKeysToMatchAdded(t *testing.T) {
diff --git a/pkg/epp/datalayer/collector_test.go b/pkg/epp/datalayer/collector_test.go
index 0e3b915..f0655a7 100644
--- a/pkg/epp/datalayer/collector_test.go
+++ b/pkg/epp/datalayer/collector_test.go
@@ -36,6 +36,7 @@ type DummySource struct {
 }
 
 func (d *DummySource) Name() string                   { return "test-dummy-data-source" }
+func (d *DummySource) Extractors() []string           { return []string{} }
 func (d *DummySource) AddExtractor(_ Extractor) error { return nil }
 func (d *DummySource) Collect(ctx context.Context, ep Endpoint) error {
 	atomic.AddInt64(&d.callCount, 1)
@@ -43,7 +44,6 @@ func (d *DummySource) Collect(ctx context.Context, ep Endpoint) error {
 }
 
 func defaultEndpoint() Endpoint {
-	ms := NewEndpoint()
 	pod := &PodInfo{
 		NamespacedName: types.NamespacedName{
 			Name:      "pod-name",
@@ -51,7 +51,7 @@ func defaultEndpoint() Endpoint {
 		},
 		Address: "1.2.3.4:5678",
 	}
-	ms.UpdatePod(pod)
+	ms := NewEndpoint(pod, nil)
 	return ms
 }
 
diff --git a/pkg/epp/datalayer/datasource_test.go b/pkg/epp/datalayer/datasource_test.go
index 90919cc..e5cb41d 100644
--- a/pkg/epp/datalayer/datasource_test.go
+++ b/pkg/epp/datalayer/datasource_test.go
@@ -29,6 +29,7 @@ type mockDataSource struct {
 }
 
 func (m *mockDataSource) Name() string                                { return m.name }
+func (m *mockDataSource) Extractors() []string                        { return []string{} }
 func (m *mockDataSource) AddExtractor(_ Extractor) error              { return nil }
 func (m *mockDataSource) Collect(_ context.Context, _ Endpoint) error { return nil }
 
@@ -43,6 +44,9 @@ func TestRegisterAndGetSource(t *testing.T) {
 	err = reg.Register(ds)
 	assert.Error(t, err, "expected error on duplicate registration")
 
+	err = reg.Register(nil)
+	assert.Error(t, err, "expected error on nil")
+
 	// Get by name
 	got, found := reg.GetNamedSource("test")
 	assert.True(t, found, "expected to find registered data source")
@@ -52,6 +56,20 @@ func TestRegisterAndGetSource(t *testing.T) {
 	all := reg.GetSources()
 	assert.Len(t, all, 1)
 	assert.Equal(t, "test", all[0].Name())
+
+	// Default registry
+	err = RegisterSource(ds)
+	assert.NoError(t, err, "expected no error on registration")
+
+	// Get by name
+	got, found = GetNamedSource[*mockDataSource]("test")
+	assert.True(t, found, "expected to find registered data source")
+	assert.Equal(t, "test", got.Name())
+
+	// Get all sources
+	all = GetSources()
+	assert.Len(t, all, 1)
+	assert.Equal(t, "test", all[0].Name())
 }
 
 func TestGetNamedSourceWhenNotFound(t *testing.T) {
diff --git a/pkg/epp/datalayer/factory_test.go b/pkg/epp/datalayer/factory_test.go
new file mode 100644
index 0000000..35cce88
--- /dev/null
+++ b/pkg/epp/datalayer/factory_test.go
@@ -0,0 +1,65 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package datalayer
+
+import (
+	"context"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"k8s.io/apimachinery/pkg/types"
+)
+
+func TestFactory(t *testing.T) {
+	source := &DummySource{}
+	factory := NewEndpointFactory([]DataSource{source}, 100*time.Millisecond)
+
+	pod1 := &PodInfo{
+		NamespacedName: types.NamespacedName{
+			Name:      "pod1",
+			Namespace: "default",
+		},
+		Address: "1.2.3.4:5678",
+	}
+	endpoint1 := factory.NewEndpoint(context.Background(), pod1, nil)
+	assert.NotNil(t, endpoint1, "failed to create endpoint")
+
+	dup := factory.NewEndpoint(context.Background(), pod1, nil)
+	assert.Nil(t, dup, "expected to fail to create a duplicate collector")
+
+	pod2 := &PodInfo{
+		NamespacedName: types.NamespacedName{
+			Name:      "pod2",
+			Namespace: "default",
+		},
+		Address: "1.2.3.4:5679",
+	}
+	endpoint2 := factory.NewEndpoint(context.Background(), pod2, nil)
+	assert.NotNil(t, endpoint2, "failed to create endpoint")
+
+	factory.ReleaseEndpoint(endpoint1)
+
+	// use Eventually for async processing
+	require.Eventually(t, func() bool {
+		return atomic.LoadInt64(&source.callCount) == 2
+	}, 290*time.Millisecond, 2*time.Millisecond, "expected 2 collections")
+
+	factory.Shutdown()
+}
diff --git a/pkg/epp/datalayer/metrics/datasource_test.go b/pkg/epp/datalayer/metrics/datasource_test.go
new file mode 100644
index 0000000..7c29375
--- /dev/null
+++ b/pkg/epp/datalayer/metrics/datasource_test.go
@@ -0,0 +1,65 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package metrics
+
+import (
+	"context"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"k8s.io/apimachinery/pkg/types"
+
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
+)
+
+func TestDatasource(t *testing.T) {
+	source := NewDataSource("https", "/metrics", true, nil)
+	extractor, err := NewExtractor(defaultTotalQueuedRequestsMetric, "", "", "")
+	assert.Nil(t, err, "failed to create extractor")
+
+	name := source.Name()
+	assert.Equal(t, DataSourceName, name)
+
+	err = source.AddExtractor(extractor)
+	assert.Nil(t, err, "failed to add extractor")
+
+	err = source.AddExtractor(extractor)
+	assert.NotNil(t, err, "expected to fail to add the same extractor twice")
+
+	extractors := source.Extractors()
+	assert.Len(t, extractors, 1)
+	assert.Equal(t, extractor.Name(), extractors[0])
+
+	err = datalayer.RegisterSource(source)
+	assert.Nil(t, err, "failed to register")
+
+	ctx := context.Background()
+	factory := datalayer.NewEndpointFactory([]datalayer.DataSource{source}, 100*time.Millisecond)
+	pod := &datalayer.PodInfo{
+		NamespacedName: types.NamespacedName{
+			Name:      "pod1",
+			Namespace: "default",
+		},
+		Address: "1.2.3.4:5678",
+	}
+	endpoint := factory.NewEndpoint(ctx, pod, nil)
+	assert.NotNil(t, endpoint, "failed to create endpoint")
+
+	err = source.Collect(ctx, endpoint)
+	assert.NotNil(t, err, "expected to fail to collect metrics")
+}
diff --git a/pkg/epp/datalayer/metrics/extractor_test.go b/pkg/epp/datalayer/metrics/extractor_test.go
new file mode 100644
index 0000000..d8aaca5
--- /dev/null
+++ b/pkg/epp/datalayer/metrics/extractor_test.go
@@ -0,0 +1,192 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package metrics
+
+import (
+	"context"
+	"testing"
+
+	"github.com/google/go-cmp/cmp"
+	dto "github.com/prometheus/client_model/go"
+	"google.golang.org/protobuf/proto"
+	"k8s.io/utils/ptr"
+
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
+)
+
+const (
+	// use hardcoded values - importing causes cycle
+	defaultTotalQueuedRequestsMetric    = "vllm:num_requests_waiting"
+	defaultKvCacheUsagePercentageMetric = "vllm:gpu_cache_usage_perc"
+	defaultLoraInfoMetric               = "vllm:lora_requests_info"
+	defaultCacheInfoMetric              = "vllm:cache_config_info"
+)
+
+func TestExtractorExtract(t *testing.T) {
+	ctx := context.Background()
+
+	if _, err := NewExtractor("vllm: dummy", "", "", ""); err == nil {
+		t.Error("expected to fail to create extractor with invalid specification")
+	}
+
+	extractor, err := NewExtractor(defaultTotalQueuedRequestsMetric,
+		defaultKvCacheUsagePercentageMetric, defaultLoraInfoMetric, defaultCacheInfoMetric)
+	if err != nil {
+		t.Fatalf("failed to create extractor: %v", err)
+	}
+
+	if name := extractor.Name(); name == "" {
+		t.Error("empty extractor name")
+	}
+
+	if inputType := extractor.ExpectedInputType(); inputType != PrometheusMetricType {
+		t.Errorf("incorrect expected input type: %v", inputType)
+	}
+
+	ep := datalayer.NewEndpoint(nil, nil)
+	if ep == nil {
+		t.Fatal("expected non-nil endpoint")
+	}
+
+	tests := []struct {
+		name    string
+		data    any
+		wantErr bool
+		updated bool // whether metrics are expected to change
+	}{
+		{
+			name:    "nil data",
+			data:    nil,
+			wantErr: true,
+			updated: false,
+		},
+		{
+			name:    "empty PrometheusMetricMap",
+			data:    PrometheusMetricMap{},
+			wantErr: true,  // errors when metrics are missing
+			updated: false, // and also not updated...
+		},
+		{
+			name: "single valid metric",
+			data: PrometheusMetricMap{
+				defaultTotalQueuedRequestsMetric: &dto.MetricFamily{
+					Type: dto.MetricType_GAUGE.Enum(),
+					Metric: []*dto.Metric{
+						{
+							Gauge: &dto.Gauge{Value: ptr.To(5.0)},
+						},
+					},
+				},
+			},
+			wantErr: true, // missing metrics can return an error
+			updated: true, // but should still update
+		},
+		{
+			name: "multiple valid metrics",
+			data: PrometheusMetricMap{
+				defaultTotalQueuedRequestsMetric: &dto.MetricFamily{
+					Type: dto.MetricType_GAUGE.Enum(),
+					Metric: []*dto.Metric{
+						{
+							Gauge: &dto.Gauge{Value: ptr.To(5.0)},
+						},
+					},
+				},
+				defaultKvCacheUsagePercentageMetric: &dto.MetricFamily{
+					Type: dto.MetricType_GAUGE.Enum(),
+					Metric: []*dto.Metric{
+						{
+							Gauge: &dto.Gauge{Value: ptr.To(0.5)},
+						},
+					},
+				},
+				defaultLoraInfoMetric: &dto.MetricFamily{
+					Type: dto.MetricType_GAUGE.Enum(),
+					Metric: []*dto.Metric{
+						{
+							Label: []*dto.LabelPair{
+								{
+									Name:  proto.String(LoraInfoRunningAdaptersMetricName),
+									Value: proto.String("lora1"),
+								},
+								{
+									Name:  proto.String(LoraInfoWaitingAdaptersMetricName),
+									Value: proto.String("lora2"),
+								},
+								{
+									Name:  proto.String(LoraInfoMaxAdaptersMetricName),
+									Value: proto.String("1"),
+								},
+							},
+						},
+					},
+				},
+				defaultCacheInfoMetric: &dto.MetricFamily{
+					Type: dto.MetricType_GAUGE.Enum(),
+					Metric: []*dto.Metric{
+						{
+							Label: []*dto.LabelPair{
+								{
+									Name:  proto.String(CacheConfigBlockSizeInfoMetricName),
+									Value: proto.String("16"),
+								},
+								{
+									Name:  proto.String(CacheConfigNumGPUBlocksMetricName),
+									Value: proto.String("1024"),
+								},
+							},
+							Gauge: &dto.Gauge{Value: ptr.To(1.0)},
+						},
+					},
+				},
+			},
+			wantErr: false,
+			updated: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			defer func() {
+				if r := recover(); r != nil {
+					t.Errorf("Extract panicked: %v", r)
+				}
+			}()
+
+			before := ep.GetMetrics().Clone()
+			err := extractor.Extract(ctx, tt.data, ep)
+			after := ep.GetMetrics()
+
+			if tt.wantErr && err == nil {
+				t.Errorf("expected error but got nil")
+			}
+			if !tt.wantErr && err != nil {
+				t.Errorf("unexpected error: %v", err)
+			}
+
+			if tt.updated {
+				if diff := cmp.Diff(before, after); diff == "" {
+					t.Errorf("expected metrics to be updated, but no change detected")
+				}
+			} else {
+				if diff := cmp.Diff(before, after); diff != "" {
+					t.Errorf("expected no metrics update, but got changes:\n%s", diff)
+				}
+			}
+		})
+	}
+}
diff --git a/pkg/epp/datalayer/metrics/logger_test.go b/pkg/epp/datalayer/metrics/logger_test.go
new file mode 100644
index 0000000..3ba2e7e
--- /dev/null
+++ b/pkg/epp/datalayer/metrics/logger_test.go
@@ -0,0 +1,124 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package metrics
+
+import (
+	"bytes"
+	"context"
+	"sync"
+	"testing"
+	"time"
+
+	"github.com/go-logr/logr"
+	"github.com/stretchr/testify/assert"
+	"go.uber.org/zap/zapcore"
+	"k8s.io/apimachinery/pkg/types"
+	ctrl "sigs.k8s.io/controller-runtime"
+	"sigs.k8s.io/controller-runtime/pkg/log/zap"
+	v1 "sigs.k8s.io/gateway-api-inference-extension/api/v1"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
+)
+
+// Buffer to write the logs to
+type buffer struct {
+	buf bytes.Buffer
+	mu  sync.Mutex
+}
+
+func (s *buffer) Write(p []byte) (int, error) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return s.buf.Write(p)
+}
+
+func (s *buffer) read() string {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return s.buf.String()
+}
+
+func TestLogger(t *testing.T) {
+	// Redirect the logger to a buffer
+	var b buffer
+	opts := &zap.Options{
+		DestWriter:  &b,
+		Development: true,
+		Level:       zapcore.Level(-5),
+	}
+	logger := zap.New(zap.UseFlagOptions(opts))
+	ctrl.SetLogger(logger)
+	ctx, cancel := context.WithCancel(context.Background())
+	ctx = logr.NewContext(ctx, logger)
+
+	StartMetricsLogger(ctx, &fakeDataStore{}, 100*time.Millisecond, 100*time.Millisecond)
+
+	time.Sleep(6 * time.Second)
+	cancel()
+
+	logOutput := b.read()
+	assert.Contains(t, logOutput, "Refreshing Prometheus Metrics	{\"ReadyPods\": 2}")
+	assert.Contains(t, logOutput, "Current Pods and metrics gathered	{\"Fresh metrics\": \"[Pod: {NamespacedName:default/pod1 PodName: Address:1.2.3.4:5678")
+	assert.Contains(t, logOutput, "Metrics: {ActiveModels:map[modelA:1] WaitingModels:map[modelB:2] MaxActiveModels:5")
+	assert.Contains(t, logOutput, "RunningQueueSize:3 WaitingQueueSize:7 KVCacheUsagePercent:42.5 KvCacheMaxTokenCapacity:2048")
+	assert.Contains(t, logOutput, "Pod: {NamespacedName:default/pod2 PodName: Address:1.2.3.4:5679")
+	assert.Contains(t, logOutput, "\"Stale metrics\": \"[]\"")
+}
+
+var pod1 = &datalayer.PodInfo{
+	NamespacedName: types.NamespacedName{
+		Name:      "pod1",
+		Namespace: "default",
+	},
+	Address: "1.2.3.4:5678",
+}
+var pod2 = &datalayer.PodInfo{
+	NamespacedName: types.NamespacedName{
+		Name:      "pod2",
+		Namespace: "default",
+	},
+	Address: "1.2.3.4:5679",
+}
+
+type fakeDataStore struct{}
+
+func (f *fakeDataStore) PoolGet() (*v1.InferencePool, error) {
+	return &v1.InferencePool{Spec: v1.InferencePoolSpec{TargetPorts: []v1.Port{{Number: 8000}}}}, nil
+}
+
+func (f *fakeDataStore) PodList(predicate func(datalayer.Endpoint) bool) []datalayer.Endpoint {
+	var m = &datalayer.Metrics{
+		ActiveModels:            map[string]int{"modelA": 1},
+		WaitingModels:           map[string]int{"modelB": 2},
+		MaxActiveModels:         5,
+		RunningQueueSize:        3,
+		WaitingQueueSize:        7,
+		KVCacheUsagePercent:     42.5,
+		KvCacheMaxTokenCapacity: 2048,
+		UpdateTime:              time.Now(),
+	}
+	ep1 := datalayer.NewEndpoint(pod1, m)
+	ep2 := datalayer.NewEndpoint(pod2, m)
+	pods := []datalayer.Endpoint{ep1, ep2}
+	res := []datalayer.Endpoint{}
+
+	for _, pod := range pods {
+		if predicate(pod) {
+			res = append(res, pod)
+		}
+	}
+	return res
+}
diff --git a/pkg/epp/datalayer/metrics/spec_test.go b/pkg/epp/datalayer/metrics/spec_test.go
index b4ad285..d62e9e7 100644
--- a/pkg/epp/datalayer/metrics/spec_test.go
+++ b/pkg/epp/datalayer/metrics/spec_test.go
@@ -26,6 +26,7 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 	"google.golang.org/protobuf/proto"
+	"k8s.io/utils/ptr"
 )
 
 // --- Test Helpers ---
@@ -477,3 +478,50 @@ func TestLabelsMatch(t *testing.T) {
 		})
 	}
 }
+
+func TestExtractValue(t *testing.T) {
+	tests := []struct {
+		name   string
+		metric *dto.Metric
+		want   float64
+	}{
+		{
+			name:   "nil metric",
+			metric: nil,
+			want:   0.0,
+		},
+		{
+			name: "unsupported metric type",
+			metric: &dto.Metric{
+				Summary: &dto.Summary{},
+			},
+			want: 0.0,
+		},
+		{
+			name: "gauge metric",
+			metric: &dto.Metric{
+				Gauge: &dto.Gauge{Value: ptr.To(42.5)},
+			},
+			want: 42.5,
+		},
+		{
+			name: "counter metric",
+			metric: &dto.Metric{
+				Counter: &dto.Counter{Value: ptr.To(99.9)},
+			},
+			want: 99.9,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			defer func() {
+				if r := recover(); r != nil {
+					t.Errorf("extractValue panicked for test %q: %v", tt.name, r)
+				}
+			}()
+			got := extractValue(tt.metric)
+			assert.Equal(t, tt.want, got)
+		})
+	}
+}
diff --git a/pkg/epp/flowcontrol/controller/controller_test.go b/pkg/epp/flowcontrol/controller/controller_test.go
index 917a5a1..9f46374 100644
--- a/pkg/epp/flowcontrol/controller/controller_test.go
+++ b/pkg/epp/flowcontrol/controller/controller_test.go
@@ -1128,7 +1128,7 @@ func setupRegistryForConcurrency(t *testing.T, numShards int, flowKey types.Flow
 			IntraFlowDispatchPolicyFunc: func(_ types.FlowKey) (framework.IntraFlowDispatchPolicy, error) {
 				return &frameworkmocks.MockIntraFlowDispatchPolicy{
 					SelectItemFunc: func(qa framework.FlowQueueAccessor) (types.QueueItemAccessor, error) {
-						return qa.PeekHead()
+						return qa.PeekHead(), nil
 					},
 				}, nil
 			},
diff --git a/pkg/epp/flowcontrol/controller/internal/processor_test.go b/pkg/epp/flowcontrol/controller/internal/processor_test.go
index 73fc5b1..ff75130 100644
--- a/pkg/epp/flowcontrol/controller/internal/processor_test.go
+++ b/pkg/epp/flowcontrol/controller/internal/processor_test.go
@@ -282,7 +282,7 @@ func (h *testHarness) intraFlowDispatchPolicy(types.FlowKey) (framework.IntraFlo
 
 	// Otherwise, use a default implementation that selects the head of the queue.
 	policy.SelectItemFunc = func(fqa framework.FlowQueueAccessor) (types.QueueItemAccessor, error) {
-		return fqa.PeekHead()
+		return fqa.PeekHead(), nil
 	}
 	return policy, nil
 }
@@ -480,11 +480,23 @@ func TestShardProcessor(t *testing.T) {
 			defer h.Stop()
 			h.Go()
 
-			// Wait for the dispatch cycle to select our item and pause inside our mock `RemoveFunc`.
-			select {
-			case <-itemIsBeingDispatched:
-			case <-time.After(testWaitTimeout):
-				t.Fatal("Timed out waiting for item to be dispatched")
+			// Advance the test clock in small increments until the item is being dispatched or timeout
+			// This is a more reliable way to ensure the processor has started and run the dispatch cycle
+			timeout := time.After(testWaitTimeout)
+			ticker := time.NewTicker(1 * time.Millisecond)
+			defer ticker.Stop()
+
+			dispatched := false
+			for !dispatched {
+				select {
+				case <-itemIsBeingDispatched:
+					dispatched = true
+				case <-timeout:
+					t.Fatal("Timed out waiting for item to be dispatched")
+				case <-ticker.C:
+					// Advance the test clock to trigger the dispatch ticker
+					h.clock.Step(1 * time.Millisecond)
+				}
 			}
 
 			// 3. The dispatch goroutine is now paused. We can now safely win the "race" by running cleanup logic.
diff --git a/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead_test.go b/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead_test.go
index 5971d3f..2885603 100644
--- a/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead_test.go
+++ b/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/besthead/besthead_test.go
@@ -17,7 +17,6 @@ limitations under the License.
 package besthead
 
 import (
-	"errors"
 	"testing"
 	"time"
 
@@ -105,10 +104,10 @@ func TestBestHead_SelectQueue(t *testing.T) {
 		ComparatorV: newTestComparator(),
 	}
 	queueEmpty := &frameworkmocks.MockFlowQueueAccessor{
-		LenV:         0,
-		PeekHeadErrV: framework.ErrQueueEmpty,
-		FlowKeyV:     types.FlowKey{ID: "flowEmpty"},
-		ComparatorV:  newTestComparator(),
+		LenV:        0,
+		PeekHeadV:   nil,
+		FlowKeyV:    types.FlowKey{ID: "flowEmpty"},
+		ComparatorV: newTestComparator(),
 	}
 
 	testCases := []struct {
@@ -151,19 +150,6 @@ func TestBestHead_SelectQueue(t *testing.T) {
 			),
 			expectedErr: framework.ErrIncompatiblePriorityType,
 		},
-		{
-			name: "QueuePeekHeadErrors",
-			band: newTestBand(
-				&frameworkmocks.MockFlowQueueAccessor{
-					LenV:         1,
-					PeekHeadErrV: errors.New("peek error"),
-					FlowKeyV:     flow1Key,
-					ComparatorV:  newTestComparator(),
-				},
-				queue2,
-			),
-			expectedQueueID: flow2ID,
-		},
 		{
 			name: "QueueComparatorIsNil",
 			band: newTestBand(
@@ -195,10 +181,10 @@ func TestBestHead_SelectQueue(t *testing.T) {
 			band: newTestBand(
 				queueEmpty,
 				&frameworkmocks.MockFlowQueueAccessor{
-					LenV:         0,
-					PeekHeadErrV: framework.ErrQueueEmpty,
-					FlowKeyV:     types.FlowKey{ID: "flowEmpty2"},
-					ComparatorV:  newTestComparator(),
+					LenV:        0,
+					PeekHeadV:   nil,
+					FlowKeyV:    types.FlowKey{ID: "flowEmpty2"},
+					ComparatorV: newTestComparator(),
 				},
 			),
 		},
diff --git a/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/functional_test.go b/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/functional_test.go
index 7649b32..3dba8b7 100644
--- a/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/functional_test.go
+++ b/pkg/epp/flowcontrol/framework/plugins/policies/interflow/dispatch/functional_test.go
@@ -63,9 +63,9 @@ func runSelectQueueConformanceTests(t *testing.T, policy framework.InterFlowDisp
 
 	flowIDEmpty := "flow-empty"
 	mockQueueEmpty := &frameworkmocks.MockFlowQueueAccessor{
-		LenV:         0,
-		PeekHeadErrV: framework.ErrQueueEmpty,
-		FlowKeyV:     types.FlowKey{ID: flowIDEmpty},
+		LenV:      0,
+		PeekHeadV: nil,
+		FlowKeyV:  types.FlowKey{ID: flowIDEmpty},
 	}
 
 	testCases := []struct {
diff --git a/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/functional_test.go b/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/functional_test.go
index 01e8a1c..4088b4d 100644
--- a/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/functional_test.go
+++ b/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/functional_test.go
@@ -22,7 +22,6 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/framework"
 	frameworkmocks "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/framework/mocks"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch"
 	_ "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/flowcontrol/framework/plugins/policies/intraflow/dispatch/fcfs"
@@ -66,8 +65,8 @@ func TestIntraFlowDispatchPolicyConformance(t *testing.T) {
 			t.Run("SelectItemFromEmptyQueue", func(t *testing.T) {
 				t.Parallel()
 				mockQueue := &frameworkmocks.MockFlowQueueAccessor{
-					PeekHeadErrV: framework.ErrQueueEmpty,
-					LenV:         0,
+					PeekHeadV: nil,
+					LenV:      0,
 				}
 				item, err := policy.SelectItem(mockQueue)
 				require.NoError(t, err, "SelectItem from an empty queue for %s should not return an error", policyName)
diff --git a/pkg/epp/flowcontrol/framework/plugins/queue/benchmark_test.go b/pkg/epp/flowcontrol/framework/plugins/queue/benchmark_test.go
index 24f4a92..b9461eb 100644
--- a/pkg/epp/flowcontrol/framework/plugins/queue/benchmark_test.go
+++ b/pkg/epp/flowcontrol/framework/plugins/queue/benchmark_test.go
@@ -70,11 +70,8 @@ func benchmarkAddRemove(b *testing.B, q framework.SafeQueue) {
 	b.RunParallel(func(pb *testing.PB) {
 		for pb.Next() {
 			item := mocks.NewMockQueueItemAccessor(1, "item", benchmarkFlowKey)
-			err := q.Add(item)
-			if err != nil {
-				b.Fatalf("Add failed: %v", err)
-			}
-			_, err = q.Remove(item.Handle())
+			q.Add(item)
+			_, err := q.Remove(item.Handle())
 			if err != nil {
 				b.Fatalf("Remove failed: %v", err)
 			}
@@ -87,27 +84,21 @@ func benchmarkAddRemove(b *testing.B, q framework.SafeQueue) {
 func benchmarkAddPeekRemove(b *testing.B, q framework.SafeQueue) {
 	// Pre-add one item so PeekHead doesn't fail on the first iteration.
 	initialItem := mocks.NewMockQueueItemAccessor(1, "initial", benchmarkFlowKey)
-	if err := q.Add(initialItem); err != nil {
-		b.Fatalf("Failed to add initial item: %v", err)
-	}
+	q.Add(initialItem)
 
 	b.ReportAllocs()
 
 	for b.Loop() {
 		item := mocks.NewMockQueueItemAccessor(1, "item", benchmarkFlowKey)
-		err := q.Add(item)
-		if err != nil {
-			b.Fatalf("Add failed: %v", err)
-		}
-
-		peeked, err := q.PeekHead()
-		if err != nil {
+		q.Add(item)
+		peeked := q.PeekHead()
+		if peeked == nil {
 			// In a concurrent benchmark, this could happen if the queue becomes empty.
 			// In a serial one, it's a fatal error.
-			b.Fatalf("PeekHead failed: %v", err)
+			b.Fatal("PeekHead failed")
 		}
 
-		_, err = q.Remove(peeked.Handle())
+		_, err := q.Remove(peeked.Handle())
 		if err != nil {
 			b.Fatalf("Remove failed: %v", err)
 		}
@@ -125,16 +116,14 @@ func benchmarkBulkAddThenBulkRemove(b *testing.B, q framework.SafeQueue) {
 		for j := range items {
 			item := mocks.NewMockQueueItemAccessor(1, fmt.Sprintf("bulk-%d-%d", i, j), benchmarkFlowKey)
 			items[j] = item
-			if err := q.Add(item); err != nil {
-				b.Fatalf("Add failed: %v", err)
-			}
+			q.Add(item)
 		}
 
 		// Remove the same number of items
 		for range items {
-			peeked, err := q.PeekHead()
-			if err != nil {
-				b.Fatalf("PeekHead failed: %v", err)
+			peeked := q.PeekHead()
+			if peeked == nil {
+				b.Fatal("PeekHead failed")
 			}
 			if _, err := q.Remove(peeked.Handle()); err != nil {
 				b.Fatalf("Remove failed: %v", err)
@@ -148,25 +137,20 @@ func benchmarkBulkAddThenBulkRemove(b *testing.B, q framework.SafeQueue) {
 func benchmarkAddPeekTailRemove(b *testing.B, q framework.SafeQueue) {
 	// Pre-add one item so PeekTail doesn't fail on the first iteration.
 	initialItem := mocks.NewMockQueueItemAccessor(1, "initial", benchmarkFlowKey)
-	if err := q.Add(initialItem); err != nil {
-		b.Fatalf("Failed to add initial item: %v", err)
-	}
+	q.Add(initialItem)
 
 	b.ReportAllocs()
 
 	for b.Loop() {
 		item := mocks.NewMockQueueItemAccessor(1, "item", benchmarkFlowKey)
-		err := q.Add(item)
-		if err != nil {
-			b.Fatalf("Add failed: %v", err)
-		}
+		q.Add(item)
 
-		peeked, err := q.PeekTail()
-		if err != nil {
-			b.Fatalf("PeekTail failed: %v", err)
+		peeked := q.PeekTail()
+		if peeked == nil {
+			b.Fatal("PeekTail failed")
 		}
 
-		_, err = q.Remove(peeked.Handle())
+		_, err := q.Remove(peeked.Handle())
 		if err != nil {
 			b.Fatalf("Remove failed: %v", err)
 		}
@@ -179,9 +163,7 @@ func benchmarkHighContention(b *testing.B, q framework.SafeQueue) {
 	// Pre-fill the queue to ensure consumers have work to do immediately.
 	for i := range 1000 {
 		item := mocks.NewMockQueueItemAccessor(1, fmt.Sprintf("prefill-%d", i), benchmarkFlowKey)
-		if err := q.Add(item); err != nil {
-			b.Fatalf("Failed to pre-fill queue: %v", err)
-		}
+		q.Add(item)
 	}
 
 	stopCh := make(chan struct{})
@@ -198,7 +180,7 @@ func benchmarkHighContention(b *testing.B, q framework.SafeQueue) {
 					return
 				default:
 					item := mocks.NewMockQueueItemAccessor(1, "item", benchmarkFlowKey)
-					_ = q.Add(item)
+					q.Add(item)
 				}
 			}
 		}()
@@ -210,8 +192,8 @@ func benchmarkHighContention(b *testing.B, q framework.SafeQueue) {
 	// Consumers drive the benchmark.
 	b.RunParallel(func(pb *testing.PB) {
 		for pb.Next() {
-			peeked, err := q.PeekHead()
-			if err == nil {
+			peeked := q.PeekHead()
+			if peeked != nil {
 				_, _ = q.Remove(peeked.Handle())
 			}
 		}
diff --git a/pkg/epp/flowcontrol/framework/plugins/queue/functional_test.go b/pkg/epp/flowcontrol/framework/plugins/queue/functional_test.go
index d632c1d..2282b8e 100644
--- a/pkg/epp/flowcontrol/framework/plugins/queue/functional_test.go
+++ b/pkg/epp/flowcontrol/framework/plugins/queue/functional_test.go
@@ -74,22 +74,16 @@ func testLifecycleAndOrdering(
 	t.Helper()
 
 	// PeekHead/PeekTail on empty queue
-	peeked, err := q.PeekHead()
-	assert.ErrorIs(t, err, framework.ErrQueueEmpty,
-		"[%s] PeekHead on empty queue should return ErrQueueEmpty", comparatorName)
+	peeked := q.PeekHead()
 	assert.Nil(t, peeked, "[%s] PeekHead on empty queue should return a nil item", comparatorName)
-	peeked, err = q.PeekTail()
-	assert.ErrorIs(t, err, framework.ErrQueueEmpty,
-		"[%s] PeekTail on empty queue should return ErrQueueEmpty", comparatorName)
+	peeked = q.PeekTail()
 	assert.Nil(t, peeked, "[%s] PeekTail on empty queue should return a nil item", comparatorName)
 
 	// Add items
 	currentExpectedLen := 0
 	var currentExpectedByteSize uint64
 	for i, item := range itemsInOrder {
-		addErr := q.Add(item)
-		require.NoError(t, addErr, "[%s] Add should not fail for a valid item (item %d, ID: %s)",
-			comparatorName, i, item.OriginalRequest().ID())
+		q.Add(item)
 		require.NotNil(t, item.Handle(), "[%s] Add must assign a non-nil handle to the item (item %d, ID: %s)",
 			comparatorName, i, item.OriginalRequest().ID())
 		require.False(t, item.Handle().IsInvalidated(),
@@ -121,8 +115,7 @@ func testLifecycleAndOrdering(
 	expectedByteSize := expectedTotalByteSize
 	for i, expectedItem := range itemsInOrder {
 		// Verify PeekHead
-		peeked, err = q.PeekHead()
-		require.NoError(t, err, "[%s] PeekHead should not error on a non-empty queue (iteration %d)", comparatorName, i)
+		peeked = q.PeekHead()
 		require.NotNil(t, peeked, "[%s] PeekHead should return a non-nil item (iteration %d)", comparatorName, i)
 		assert.Equal(t, expectedItem.OriginalRequest().ID(), peeked.OriginalRequest().ID(),
 			"[%s] PeekHead must return the item (ID: %s) at the head of the queue (iteration %d)",
@@ -137,8 +130,7 @@ func testLifecycleAndOrdering(
 			"[%s] ByteSize() must be unchanged after PeekHead (iteration %d)", comparatorName, i)
 
 		// Verify PeekTail
-		peekedTail, err := q.PeekTail()
-		require.NoError(t, err, "[%s] PeekTail should not error on a non-empty queue (iteration %d)", comparatorName, i)
+		peekedTail := q.PeekTail()
 		require.NotNil(t, peekedTail, "[%s] PeekTail should return a non-nil item (iteration %d)", comparatorName, i)
 		// The tail is the last item in the *remaining* ordered slice.
 		expectedTailItem := itemsInOrder[len(itemsInOrder)-1]
@@ -166,9 +158,7 @@ func testLifecycleAndOrdering(
 	assert.Zero(t, q.Len(), "[%s] Queue length should be 0 after all items are removed", comparatorName)
 	assert.Zero(t, q.ByteSize(), "[%s] Queue byte size should be 0 after all items are removed", comparatorName)
 
-	peeked, err = q.PeekHead()
-	assert.ErrorIs(t, err, framework.ErrQueueEmpty, "[%s] PeekHead on an empty queue should return ErrQueueEmpty again",
-		comparatorName)
+	peeked = q.PeekHead()
 	assert.Nil(t, peeked, "[%s] PeekHead on an empty queue should return a nil item again", comparatorName)
 }
 
@@ -247,33 +237,18 @@ func TestQueueConformance(t *testing.T) {
 				})
 			}
 
-			t.Run("Add_NilItem", func(t *testing.T) {
-				t.Parallel()
-				q, err := constructor(enqueueTimeComparator)
-				require.NoError(t, err, "Setup: creating queue for test should not fail")
-
-				currentLen := q.Len()
-				currentByteSize := q.ByteSize()
-				err = q.Add(nil)
-				assert.ErrorIs(t, err, framework.ErrNilQueueItem, "Add(nil) must return ErrNilQueueItem")
-				assert.Equal(t, currentLen, q.Len(), "The queue's length must not change after a failed Add")
-				assert.Equal(t, currentByteSize, q.ByteSize(), "The queue's byte size must not change after a failed Add")
-			})
-
 			t.Run("Remove_InvalidHandle", func(t *testing.T) {
 				t.Parallel()
 				q, err := constructor(enqueueTimeComparator)
 				require.NoError(t, err, "Setup: creating queue for test should not fail")
 
 				item := typesmocks.NewMockQueueItemAccessor(100, "item", flowKey)
-				err = q.Add(item)
-				require.NoError(t, err, "Setup: adding an item should succeed")
+				q.Add(item)
 
 				otherQ, err := constructor(enqueueTimeComparator) // A different queue instance
 				require.NoError(t, err, "Setup: creating otherQ should succeed")
 				otherItem := typesmocks.NewMockQueueItemAccessor(10, "other_item", types.FlowKey{ID: "other-flow"})
-				err = otherQ.Add(otherItem)
-				require.NoError(t, err, "Setup: adding item to otherQ should succeed")
+				otherQ.Add(otherItem)
 				alienHandle := otherItem.Handle()
 				require.NotNil(t, alienHandle, "Setup: alien handle should not be nil")
 
@@ -322,9 +297,9 @@ func TestQueueConformance(t *testing.T) {
 				item3 := typesmocks.NewMockQueueItemAccessor(30, "item3_nonhead", flowKey)
 				item3.EnqueueTimeV = now.Add(-1 * time.Second)
 
-				_ = q.Add(item1)
-				_ = q.Add(item2)
-				_ = q.Add(item3)
+				q.Add(item1)
+				q.Add(item2)
+				q.Add(item3)
 				require.Equal(t, 3, q.Len(), "Queue should have 3 items before removing non-head")
 				handleNonHead := item2.Handle()
 
@@ -351,8 +326,7 @@ func TestQueueConformance(t *testing.T) {
 			t.Run("Cleanup_EmptyQueue", func(t *testing.T) {
 				t.Parallel()
 				emptyQ, _ := constructor(enqueueTimeComparator)
-				cleanedItems, err := emptyQ.Cleanup(predicateRemoveOddSizes)
-				require.NoError(t, err, "Cleanup on an empty queue should not return an error")
+				cleanedItems := emptyQ.Cleanup(predicateRemoveOddSizes)
 				assert.Empty(t, cleanedItems, "Cleanup on an empty queue should return an empty slice")
 				assert.Zero(t, emptyQ.Len(), "Len() should be 0 after Cleanup on an empty queue")
 				assert.Zero(t, emptyQ.ByteSize(), "ByteSize() should be 0 after Cleanup on an empty queue")
@@ -363,13 +337,12 @@ func TestQueueConformance(t *testing.T) {
 				q, _ := constructor(enqueueTimeComparator)
 				itemK1 := typesmocks.NewMockQueueItemAccessor(10, "k1_matchNone", flowKey)
 				itemK2 := typesmocks.NewMockQueueItemAccessor(12, "k2_matchNone", flowKey)
-				_ = q.Add(itemK1)
-				_ = q.Add(itemK2)
+				q.Add(itemK1)
+				q.Add(itemK2)
 				initialLen := q.Len()
 				initialBs := q.ByteSize()
 
-				cleanedItems, err := q.Cleanup(func(item types.QueueItemAccessor) bool { return false })
-				require.NoError(t, err, "Cleanup should not return an error")
+				cleanedItems := q.Cleanup(func(item types.QueueItemAccessor) bool { return false })
 				assert.Empty(t, cleanedItems, "Cleanup should return an empty slice when no items match the predicate")
 				assert.Equal(t, initialLen, q.Len(), "Len() should not change after Cleanup when no items match thepredicate")
 				assert.Equal(t, initialBs, q.ByteSize(),
@@ -383,11 +356,10 @@ func TestQueueConformance(t *testing.T) {
 				q, _ := constructor(enqueueTimeComparator)
 				itemR1 := typesmocks.NewMockQueueItemAccessor(11, "r1_matchAll", flowKey)
 				itemR2 := typesmocks.NewMockQueueItemAccessor(13, "r2_matchAll", flowKey)
-				_ = q.Add(itemR1)
-				_ = q.Add(itemR2)
+				q.Add(itemR1)
+				q.Add(itemR2)
 
-				cleanedItems, err := q.Cleanup(func(item types.QueueItemAccessor) bool { return true })
-				require.NoError(t, err, "Cleanup should not return an error")
+				cleanedItems := q.Cleanup(func(item types.QueueItemAccessor) bool { return true })
 				assert.Len(t, cleanedItems, 2, "Cleanup should return all items that matched the predicate")
 				assert.Zero(t, q.Len(), "Len() should be 0 after Cleanup")
 				assert.Zero(t, q.ByteSize(), "ByteSize() should be 0 after Cleanup")
@@ -402,15 +374,14 @@ func TestQueueConformance(t *testing.T) {
 				iR1 := typesmocks.NewMockQueueItemAccessor(11, "r1_subset", flowKey)
 				iK2 := typesmocks.NewMockQueueItemAccessor(22, "k2_subset", flowKey)
 				iR2 := typesmocks.NewMockQueueItemAccessor(33, "r2_subset", flowKey)
-				_ = q.Add(iK1)
-				_ = q.Add(iR1)
-				_ = q.Add(iK2)
-				_ = q.Add(iR2)
+				q.Add(iK1)
+				q.Add(iR1)
+				q.Add(iK2)
+				q.Add(iR2)
 
 				expectedKeptByteSize := iK1.OriginalRequest().ByteSize() + iK2.OriginalRequest().ByteSize()
 
-				cleanedItems, err := q.Cleanup(predicateRemoveOddSizes)
-				require.NoError(t, err, "Cleanup should not return an error")
+				cleanedItems := q.Cleanup(predicateRemoveOddSizes)
 				assert.Len(t, cleanedItems, 2, "Cleanup should return 2 items that matched the predicate")
 				assert.Equal(t, 2, q.Len(), "Len() should be 2 after Cleanup")
 				assert.Equal(t, expectedKeptByteSize, q.ByteSize(), "ByteSize() should be sum of kept items after Cleanup")
@@ -435,7 +406,7 @@ func TestQueueConformance(t *testing.T) {
 				// Verify remaining items are correct
 				var remainingIDs []string
 				for q.Len() > 0 {
-					peeked, _ := q.PeekHead()
+					peeked := q.PeekHead()
 					item, _ := q.Remove(peeked.Handle())
 					remainingIDs = append(remainingIDs, item.OriginalRequest().ID())
 				}
@@ -452,11 +423,10 @@ func TestQueueConformance(t *testing.T) {
 
 				itemD1 := typesmocks.NewMockQueueItemAccessor(10, "ditem1", flowKey)
 				itemD2 := typesmocks.NewMockQueueItemAccessor(20, "ditem2", flowKey)
-				_ = q.Add(itemD1)
-				_ = q.Add(itemD2)
+				q.Add(itemD1)
+				q.Add(itemD2)
 
-				drainedItems, err := q.Drain()
-				require.NoError(t, err, "Drain on a non-empty queue should not fail")
+				drainedItems := q.Drain()
 				assert.Len(t, drainedItems, 2, "Drain should return all items that were in the queue")
 				assert.Zero(t, q.Len(), "Queue length must be 0 after Drain")
 				assert.Zero(t, q.ByteSize(), "Queue byte size must be 0 after Drain")
@@ -482,12 +452,10 @@ func TestQueueConformance(t *testing.T) {
 				q, err := constructor(enqueueTimeComparator)
 				require.NoError(t, err, "Setup: creating queue for empty drain test should not fail")
 
-				drainedItems, err := q.Drain() // First drain on empty
-				require.NoError(t, err, "Drain on an empty queue should not fail")
+				drainedItems := q.Drain() // First drain on empty
 				assert.Empty(t, drainedItems, "Drain on an empty queue should return an empty slice")
 
-				drainedAgain, err := q.Drain() // Second drain on already empty
-				require.NoError(t, err, "Second drain on an already empty queue should not fail")
+				drainedAgain := q.Drain() // Second drain on already empty
 				assert.Empty(t, drainedAgain, "Second drain on an already empty queue should return an empty slice")
 				assert.Zero(t, q.Len())
 				assert.Zero(t, q.ByteSize())
@@ -510,7 +478,7 @@ func TestQueueConformance(t *testing.T) {
 				// Pre-populate the queue with an initial set of items.
 				for i := range initialItems {
 					item := typesmocks.NewMockQueueItemAccessor(1, fmt.Sprintf("%s_conc_init_%d", flowKey, i), flowKey)
-					err := q.Add(item)
+					q.Add(item)
 					require.NoError(t, err, "Setup: pre-populating the queue should not fail")
 					handleChan <- item.Handle()
 				}
@@ -529,11 +497,9 @@ func TestQueueConformance(t *testing.T) {
 							case 0: // Add
 								item := typesmocks.NewMockQueueItemAccessor(1,
 									fmt.Sprintf("%s_conc_init_%d_%d", flowKey, routineID, j), flowKey)
-								err := q.Add(item)
-								if assert.NoError(t, err, "Add must be goroutine-safe") {
-									successfulAdds.Add(1)
-									handleChan <- item.Handle()
-								}
+								q.Add(item)
+								successfulAdds.Add(1)
+								handleChan <- item.Handle()
 							case 1: // Remove
 								select {
 								case handle := <-handleChan:
@@ -553,17 +519,16 @@ func TestQueueConformance(t *testing.T) {
 							case 2: // Inspect
 								_ = q.Len()
 								_ = q.ByteSize()
-								_, err := q.PeekHead()
-								if q.Len() == 0 { // Only expect ErrQueueEmpty if Len is 0
-									assert.ErrorIs(t, err, framework.ErrQueueEmpty, "Peek on empty queue expected ErrQueueEmpty")
+								peeked := q.PeekHead()
+								if q.Len() == 0 {
+									assert.Nil(t, peeked, "PeekHead on empty queue expected nil")
 								}
-								_, err = q.PeekTail()
-								if q.Len() == 0 { // Only expect ErrQueueEmpty if Len is 0
-									assert.ErrorIs(t, err, framework.ErrQueueEmpty, "PeekTail on empty queue expected ErrQueueEmpty")
+								peeked = q.PeekTail()
+								if q.Len() == 0 {
+									assert.Nil(t, peeked, "PeekTail on empty queue expected nil")
 								}
 							case 3: // Cleanup
-								_, cleanupErr := q.Cleanup(func(item types.QueueItemAccessor) bool { return false })
-								assert.NoError(t, cleanupErr, "Cleanup (no-op) should be goroutine-safe")
+								q.Cleanup(func(item types.QueueItemAccessor) bool { return false })
 							}
 						}
 					}(i)
@@ -573,8 +538,7 @@ func TestQueueConformance(t *testing.T) {
 				close(handleChan)
 
 				// Drain the queue to verify all handles are invalidated and to count remaining items accurately.
-				drainedItems, drainErr := q.Drain()
-				require.NoError(t, drainErr, "Draining queue at the end of concurrency test should not fail")
+				drainedItems := q.Drain()
 
 				for _, item := range drainedItems {
 					require.True(t, item.Handle().IsInvalidated(), "All handles from final drain must be invalidated")
diff --git a/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap_test.go b/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap_test.go
index b4e22ab..ee6fab7 100644
--- a/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap_test.go
+++ b/pkg/epp/flowcontrol/framework/plugins/queue/maxminheap_test.go
@@ -39,8 +39,7 @@ func TestMaxMinHeap_InternalProperty(t *testing.T) {
 		// Add items in a somewhat random order of enqueue times
 		items[i] = typesmocks.NewMockQueueItemAccessor(10, "item", types.FlowKey{ID: "flow"})
 		items[i].EnqueueTimeV = now.Add(time.Duration((i%5-2)*10) * time.Second)
-		err := q.Add(items[i])
-		require.NoError(t, err, "Add should not fail")
+		q.Add(items[i])
 		assertHeapProperty(t, q, "after adding item %d", i)
 	}
 
@@ -54,9 +53,9 @@ func TestMaxMinHeap_InternalProperty(t *testing.T) {
 
 	// Remove remaining items from the head and validate each time
 	for q.Len() > 0 {
-		head, err := q.PeekHead()
-		require.NoError(t, err)
-		_, err = q.Remove(head.Handle())
+		head := q.PeekHead()
+		require.NotNil(t, head)
+		_, err := q.Remove(head.Handle())
 		require.NoError(t, err)
 		assertHeapProperty(t, q, "after removing head item")
 	}
diff --git a/pkg/epp/flowcontrol/registry/managedqueue_test.go b/pkg/epp/flowcontrol/registry/managedqueue_test.go
index b4d32fb..1e0afac 100644
--- a/pkg/epp/flowcontrol/registry/managedqueue_test.go
+++ b/pkg/epp/flowcontrol/registry/managedqueue_test.go
@@ -132,23 +132,13 @@ func TestManagedQueue_Add(t *testing.T) {
 		{
 			name: "ShouldSucceed_AndIncrementStats",
 			setupMock: func(q *frameworkmocks.MockSafeQueue) {
-				q.AddFunc = func(types.QueueItemAccessor) error { return nil }
+				q.AddFunc = func(types.QueueItemAccessor) {}
 			},
 			isDraining:            false,
 			expectErr:             false,
 			expectedLenDelta:      1,
 			expectedByteSizeDelta: 100,
 		},
-		{
-			name: "ShouldFail_AndNotChangeStats_WhenUnderlyingQueueFails",
-			setupMock: func(q *frameworkmocks.MockSafeQueue) {
-				q.AddFunc = func(types.QueueItemAccessor) error { return errors.New("add failed") }
-			},
-			isDraining:            false,
-			expectErr:             true,
-			expectedLenDelta:      0,
-			expectedByteSizeDelta: 0,
-		},
 		{
 			name:                  "ShouldFail_AndNotChangeStats_WhenQueueIsDraining",
 			setupMock:             func(q *frameworkmocks.MockSafeQueue) {},
@@ -255,40 +245,26 @@ func TestManagedQueue_Cleanup(t *testing.T) {
 	testCases := []struct {
 		name                  string
 		setupMock             func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor)
-		expectErr             bool
 		expectedLenDelta      int64
 		expectedByteSizeDelta int64
 	}{
 		{
 			name: "ShouldSucceed_AndDecrementStats_WhenItemsRemoved",
 			setupMock: func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor) {
-				q.CleanupFunc = func(_ framework.PredicateFunc) ([]types.QueueItemAccessor, error) {
-					return items, nil
+				q.CleanupFunc = func(_ framework.PredicateFunc) []types.QueueItemAccessor {
+					return items
 				}
 			},
-			expectErr:             false,
 			expectedLenDelta:      -2,
 			expectedByteSizeDelta: -125,
 		},
 		{
 			name: "ShouldSucceed_AndNotChangeStats_WhenNoItemsRemoved",
 			setupMock: func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor) {
-				q.CleanupFunc = func(_ framework.PredicateFunc) ([]types.QueueItemAccessor, error) {
-					return nil, nil // Simulate no items matching predicate.
+				q.CleanupFunc = func(_ framework.PredicateFunc) []types.QueueItemAccessor {
+					return nil // Simulate no items matching predicate.
 				}
 			},
-			expectErr:             false,
-			expectedLenDelta:      0,
-			expectedByteSizeDelta: 0,
-		},
-		{
-			name: "ShouldFail_AndNotChangeStats_WhenUnderlyingQueueFails",
-			setupMock: func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor) {
-				q.CleanupFunc = func(_ framework.PredicateFunc) ([]types.QueueItemAccessor, error) {
-					return nil, errors.New("cleanup failed")
-				}
-			},
-			expectErr:             true,
 			expectedLenDelta:      0,
 			expectedByteSizeDelta: 0,
 		},
@@ -305,14 +281,7 @@ func TestManagedQueue_Cleanup(t *testing.T) {
 			}
 			h.setupWithItems(items...)
 			tc.setupMock(q, items)
-
-			_, err := h.mq.Cleanup(func(_ types.QueueItemAccessor) bool { return true })
-
-			if tc.expectErr {
-				require.Error(t, err, "Cleanup operation must fail if the underlying queue implementation encounters an error")
-			} else {
-				require.NoError(t, err, "Cleanup operation must succeed if the underlying queue implementation succeeds")
-			}
+			h.mq.Cleanup(func(_ types.QueueItemAccessor) bool { return true })
 			assert.Equal(t, tc.expectedLenDelta, h.propagator.lenDelta.Load(),
 				"The propagated length delta must exactly match the total number of items removed during cleanup")
 			assert.Equal(t, tc.expectedByteSizeDelta, h.propagator.byteSizeDelta.Load(),
@@ -328,32 +297,19 @@ func TestManagedQueue_Drain(t *testing.T) {
 	testCases := []struct {
 		name                  string
 		setupMock             func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor)
-		expectErr             bool
 		expectedLenDelta      int64
 		expectedByteSizeDelta int64
 	}{
 		{
 			name: "ShouldSucceed_AndDecrementStats",
 			setupMock: func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor) {
-				q.DrainFunc = func() ([]types.QueueItemAccessor, error) {
-					return items, nil
+				q.DrainFunc = func() []types.QueueItemAccessor {
+					return items
 				}
 			},
-			expectErr:             false,
 			expectedLenDelta:      -2,
 			expectedByteSizeDelta: -125,
 		},
-		{
-			name: "ShouldFail_AndNotChangeStats_WhenUnderlyingQueueFails",
-			setupMock: func(q *frameworkmocks.MockSafeQueue, items []types.QueueItemAccessor) {
-				q.DrainFunc = func() ([]types.QueueItemAccessor, error) {
-					return nil, errors.New("drain failed")
-				}
-			},
-			expectErr:             true,
-			expectedLenDelta:      0,
-			expectedByteSizeDelta: 0,
-		},
 	}
 
 	for _, tc := range testCases {
@@ -368,13 +324,7 @@ func TestManagedQueue_Drain(t *testing.T) {
 			h.setupWithItems(items...)
 			tc.setupMock(q, items)
 
-			_, err := h.mq.Drain()
-
-			if tc.expectErr {
-				require.Error(t, err, "Drain operation must fail if the underlying queue implementation encounters an error")
-			} else {
-				require.NoError(t, err, "Drain operation must succeed if the underlying queue implementation succeeds")
-			}
+			h.mq.Drain()
 			assert.Equal(t, tc.expectedLenDelta, h.propagator.lenDelta.Load(),
 				"The propagated length delta must exactly match the total number of items drained")
 			assert.Equal(t, tc.expectedByteSizeDelta, h.propagator.byteSizeDelta.Load(),
@@ -401,8 +351,8 @@ func TestManagedQueue_FlowQueueAccessor(t *testing.T) {
 		accessor := harness.mq.FlowQueueAccessor()
 		require.NotNil(t, accessor, "FlowQueueAccessor must return a non-nil instance (guaranteed by contract)")
 
-		assert.Equal(t, harness.mq.Name(), accessor.Name(), "Accessor Name() must proxy the underlying queue's name")
-		assert.Equal(t, harness.mq.Capabilities(), accessor.Capabilities(),
+		assert.Equal(t, harness.mq.queue.Name(), accessor.Name(), "Accessor Name() must proxy the underlying queue's name")
+		assert.Equal(t, harness.mq.queue.Capabilities(), accessor.Capabilities(),
 			"Accessor Capabilities() must proxy the underlying queue's capabilities")
 		assert.Equal(t, harness.mq.Len(), accessor.Len(), "Accessor Len() must reflect the managed queue's current length")
 		assert.Equal(t, harness.mq.ByteSize(), accessor.ByteSize(),
@@ -410,15 +360,11 @@ func TestManagedQueue_FlowQueueAccessor(t *testing.T) {
 		assert.Equal(t, flowKey, accessor.FlowKey(), "Accessor FlowKey() must return the correct identifier for the flow")
 		assert.Equal(t, harness.mockPolicy.Comparator(), accessor.Comparator(),
 			"Accessor Comparator() must return the comparator provided by the configured intra-flow policy")
-		assert.Equal(t, harness.mockPolicy.Comparator(), harness.mq.Comparator(),
-			"ManagedQueue Comparator() must also return the comparator provided by the configured intra-flow policy")
 
-		peekedHead, err := accessor.PeekHead()
-		require.NoError(t, err, "Accessor PeekHead() must succeed when the underlying queue succeeds")
+		peekedHead := accessor.PeekHead()
 		assert.Same(t, item, peekedHead, "Accessor PeekHead() must return the exact item instance at the head")
 
-		peekedTail, err := accessor.PeekTail()
-		require.NoError(t, err, "Accessor PeekTail() must succeed when the underlying queue succeeds")
+		peekedTail := accessor.PeekTail()
 		assert.Same(t, item, peekedTail, "Accessor PeekTail() must return the exact item instance at the tail")
 	})
 
@@ -426,14 +372,10 @@ func TestManagedQueue_FlowQueueAccessor(t *testing.T) {
 		t.Parallel()
 		flowKey := types.FlowKey{ID: "flow", Priority: 1}
 		q := &frameworkmocks.MockSafeQueue{}
-		expectedErr := errors.New("queue is empty")
-		q.PeekHeadErrV = expectedErr
+		q.PeekHeadV = nil
 		harness := newMockedMqHarness(t, q, flowKey)
 		accessor := harness.mq.FlowQueueAccessor()
-
-		_, err := accessor.PeekHead()
-		require.Error(t, err, "Accessor PeekHead() should return an error on an empty queue")
-		assert.ErrorIs(t, err, expectedErr, "Accessor should proxy the specific error from the underlying queue")
+		assert.Nil(t, accessor.PeekHead(), "Accessor PeekHead() should return an nil on an empty queue")
 	})
 }
 
@@ -470,9 +412,7 @@ func TestManagedQueue_Concurrency_StatsIntegrity(t *testing.T) {
 
 	// After all operations, the queue should ideally be empty, but we drain any remaining items to get a definitive final
 	// state.
-	_, err := h.mq.Drain()
-	require.NoError(t, err, "Final drain operation must succeed to finalize the test state")
-
+	h.mq.Drain()
 	assert.Zero(t, h.mq.Len(), "Final queue length must be zero after draining all remaining items")
 	assert.Zero(t, h.mq.ByteSize(), "Final queue byte size must be zero after draining all remaining items")
 	assert.Equal(t, int64(0), h.propagator.lenDelta.Load(),
@@ -488,7 +428,7 @@ func TestManagedQueue_InvariantPanics_OnUnderflow(t *testing.T) {
 	flowKey := types.FlowKey{ID: "flow", Priority: 1}
 	item := typesmocks.NewMockQueueItemAccessor(100, "req", flowKey)
 	q := &frameworkmocks.MockSafeQueue{}
-	q.AddFunc = func(types.QueueItemAccessor) error { return nil }
+	q.AddFunc = func(types.QueueItemAccessor) {}
 	q.RemoveFunc = func(types.QueueItemHandle) (types.QueueItemAccessor, error) {
 		return item, nil
 	}
diff --git a/pkg/epp/requestcontrol/director_test.go b/pkg/epp/requestcontrol/director_test.go
index 8cb9c91..e705bee 100644
--- a/pkg/epp/requestcontrol/director_test.go
+++ b/pkg/epp/requestcontrol/director_test.go
@@ -20,6 +20,7 @@ import (
 	"context"
 	"errors"
 	"fmt"
+	"maps"
 	"testing"
 	"time"
 
@@ -37,6 +38,7 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/apix/v1alpha2"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datalayer"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datastore"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/handlers"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metadata"
@@ -48,6 +50,10 @@ import (
 	testutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/testing"
 )
 
+const (
+	mockProducedDataKey = "producedDataKey"
+)
+
 // --- Mocks ---
 
 type mockAdmissionController struct {
@@ -66,9 +72,16 @@ func (m *mockAdmissionController) Admit(
 type mockScheduler struct {
 	scheduleResults *schedulingtypes.SchedulingResult
 	scheduleErr     error
+	dataProduced    bool // denotes whether data production is expected.
 }
 
-func (m *mockScheduler) Schedule(_ context.Context, _ *schedulingtypes.LLMRequest, _ []schedulingtypes.Pod) (*schedulingtypes.SchedulingResult, error) {
+func (m *mockScheduler) Schedule(_ context.Context, _ *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) (*schedulingtypes.SchedulingResult, error) {
+	if pods != nil && m.dataProduced {
+		data, ok := pods[0].Get(mockProducedDataKey)
+		if !ok || data.(mockProducedDataType).value != 42 {
+			return nil, errors.New("expected produced data not found in pod")
+		}
+	}
 	return m.scheduleResults, m.scheduleErr
 }
 
@@ -93,6 +106,66 @@ func (ds *mockDatastore) PodList(predicate func(backendmetrics.PodMetrics) bool)
 	return res
 }
 
+type mockPrepareDataPlugin struct {
+	name     string
+	produces map[string]any
+	consumes map[string]any
+}
+
+func (m *mockPrepareDataPlugin) TypedName() plugins.TypedName {
+	return plugins.TypedName{Name: m.name, Type: "mock"}
+}
+
+func (m *mockPrepareDataPlugin) Produces() map[string]any {
+	return m.produces
+}
+
+func (m *mockPrepareDataPlugin) Consumes() map[string]any {
+	return m.consumes
+}
+
+func (m *mockPrepareDataPlugin) PrepareRequestData(ctx context.Context, request *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) error {
+	pods[0].Put(mockProducedDataKey, mockProducedDataType{value: 42})
+	return nil
+}
+
+func newMockPrepareDataPlugin(name string) *mockPrepareDataPlugin {
+	return &mockPrepareDataPlugin{
+		name:     name,
+		produces: map[string]any{mockProducedDataKey: 0},
+		consumes: map[string]any{},
+	}
+}
+
+type mockAdmissionPlugin struct {
+	tn          plugins.TypedName
+	denialError error
+}
+
+func newMockAdmissionPlugin(name string, denialError error) *mockAdmissionPlugin {
+	return &mockAdmissionPlugin{
+		tn:          plugins.TypedName{Type: "mock-admit-data", Name: name},
+		denialError: denialError,
+	}
+}
+
+func (m *mockAdmissionPlugin) TypedName() plugins.TypedName {
+	return m.tn
+}
+
+func (m *mockAdmissionPlugin) AdmitRequest(ctx context.Context, request *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) error {
+	return m.denialError
+}
+
+type mockProducedDataType struct {
+	value int
+}
+
+// Clone implements types.Cloneable.
+func (m mockProducedDataType) Clone() datalayer.Cloneable {
+	return mockProducedDataType{value: m.value}
+}
+
 func TestDirector_HandleRequest(t *testing.T) {
 	ctx := logutil.NewTestLoggerIntoContext(context.Background())
 
@@ -167,6 +240,7 @@ func TestDirector_HandleRequest(t *testing.T) {
 				TargetPods: []schedulingtypes.Pod{
 					&schedulingtypes.ScoredPod{
 						Pod: &schedulingtypes.PodMetrics{
+							AttributeMap: datalayer.NewAttributes(),
 							Pod: &backend.Pod{
 								Address:        "192.168.1.100",
 								Port:           "8000",
@@ -177,6 +251,7 @@ func TestDirector_HandleRequest(t *testing.T) {
 					},
 					&schedulingtypes.ScoredPod{
 						Pod: &schedulingtypes.PodMetrics{
+							AttributeMap: datalayer.NewAttributes(),
 							Pod: &backend.Pod{
 								Address:        "192.168.2.100",
 								Port:           "8000",
@@ -187,6 +262,7 @@ func TestDirector_HandleRequest(t *testing.T) {
 					},
 					&schedulingtypes.ScoredPod{
 						Pod: &schedulingtypes.PodMetrics{
+							AttributeMap: datalayer.NewAttributes(),
 							Pod: &backend.Pod{
 								Address:        "192.168.4.100",
 								Port:           "8000",
@@ -211,6 +287,8 @@ func TestDirector_HandleRequest(t *testing.T) {
 		wantReqCtx              *handlers.RequestContext // Fields to check in the returned RequestContext
 		wantMutatedBodyModel    string                   // Expected model in reqCtx.Request.Body after PostDispatch
 		targetModelName         string                   // Expected model name after target model resolution
+		admitRequestDenialError error                    // Expected denial error from admission plugin
+		prepareDataPlugin       *mockPrepareDataPlugin
 	}{
 		{
 			name: "successful completions request",
@@ -265,6 +343,85 @@ func TestDirector_HandleRequest(t *testing.T) {
 			wantMutatedBodyModel: model,
 			targetModelName:      model,
 		},
+		{
+			name: "successful chat completions request with prepare data plugins",
+			reqBodyMap: map[string]any{
+				"model": model,
+				"messages": []any{
+					map[string]any{
+						"role":    "user",
+						"content": "critical prompt",
+					},
+				},
+			},
+			mockAdmissionController: &mockAdmissionController{admitErr: nil},
+			schedulerMockSetup: func(m *mockScheduler) {
+				m.scheduleResults = defaultSuccessfulScheduleResults
+				m.dataProduced = true
+			},
+			wantReqCtx: &handlers.RequestContext{
+				TargetModelName: model,
+				TargetPod: &backend.Pod{
+					NamespacedName: types.NamespacedName{Namespace: "default", Name: "pod1"},
+					Address:        "192.168.1.100",
+					Port:           "8000",
+					MetricsHost:    "192.168.1.100:8000",
+				},
+				TargetEndpoint: "192.168.1.100:8000,192.168.2.100:8000,192.168.4.100:8000",
+			},
+			wantMutatedBodyModel: model,
+			targetModelName:      model,
+			prepareDataPlugin:    newMockPrepareDataPlugin("test-plugin"),
+		},
+		{
+			name: "successful chat completions request with admit request plugins",
+			reqBodyMap: map[string]any{
+				"model": model,
+				"messages": []any{
+					map[string]any{
+						"role":    "user",
+						"content": "critical prompt",
+					},
+				},
+			},
+			mockAdmissionController: &mockAdmissionController{admitErr: nil},
+			schedulerMockSetup: func(m *mockScheduler) {
+				m.scheduleResults = defaultSuccessfulScheduleResults
+			},
+			wantReqCtx: &handlers.RequestContext{
+				TargetModelName: model,
+				TargetPod: &backend.Pod{
+					NamespacedName: types.NamespacedName{Namespace: "default", Name: "pod1"},
+					Address:        "192.168.1.100",
+					Port:           "8000",
+					MetricsHost:    "192.168.1.100:8000",
+				},
+				TargetEndpoint: "192.168.1.100:8000,192.168.2.100:8000,192.168.4.100:8000",
+			},
+			wantMutatedBodyModel:    model,
+			targetModelName:         model,
+			admitRequestDenialError: nil,
+		},
+		{
+			name: "denied request by admit request plugin",
+			reqBodyMap: map[string]any{
+				"model": model,
+				"messages": []any{
+					map[string]any{
+						"role":    "user",
+						"content": "critical prompt",
+					},
+				},
+			},
+			mockAdmissionController: &mockAdmissionController{admitErr: nil},
+			schedulerMockSetup: func(m *mockScheduler) {
+				m.scheduleResults = defaultSuccessfulScheduleResults
+			},
+			wantMutatedBodyModel:    model,
+			targetModelName:         model,
+			admitRequestDenialError: errors.New("denied by admit plugin"),
+			wantErrCode:             errutil.Internal,
+		},
 		{
 			name: "successful chat completions request with multiple messages",
 			reqBodyMap: map[string]any{
@@ -414,7 +571,12 @@ func TestDirector_HandleRequest(t *testing.T) {
 			if test.schedulerMockSetup != nil {
 				test.schedulerMockSetup(mockSched)
 			}
-			director := NewDirectorWithConfig(ds, mockSched, test.mockAdmissionController, NewConfig())
+			config := NewConfig()
+			if test.prepareDataPlugin != nil {
+				config = config.WithPrepareDataPlugins(test.prepareDataPlugin)
+			}
+			config = config.WithAdmissionPlugins(newMockAdmissionPlugin("test-admit-plugin", test.admitRequestDenialError))
+			director := NewDirectorWithConfig(ds, mockSched, test.mockAdmissionController, config)
 
 			reqCtx := &handlers.RequestContext{
 				Request: &handlers.Request{
@@ -428,9 +590,7 @@ func TestDirector_HandleRequest(t *testing.T) {
 				TargetModelName: test.targetModelName,
 			}
 			// Deep copy the body map.
-			for k, v := range test.reqBodyMap {
-				reqCtx.Request.Body[k] = v
-			}
+			maps.Copy(reqCtx.Request.Body, test.reqBodyMap)
 
 			returnedReqCtx, err := director.HandleRequest(ctx, reqCtx)
 
diff --git a/pkg/epp/requestcontrol/plugin_executor_test.go b/pkg/epp/requestcontrol/plugin_executor_test.go
new file mode 100644
index 0000000..b37264f
--- /dev/null
+++ b/pkg/epp/requestcontrol/plugin_executor_test.go
@@ -0,0 +1,159 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package requestcontrol
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+	schedulingtypes "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
+)
+
+var _ PrepareDataPlugin = &mockPrepareRequestDataPlugin{}
+
+type mockPrepareRequestDataPlugin struct {
+	name      string
+	delay     time.Duration
+	returnErr error
+	executed  bool
+}
+
+func (m *mockPrepareRequestDataPlugin) TypedName() plugins.TypedName {
+	return plugins.TypedName{Type: "mock", Name: m.name}
+}
+
+func (m *mockPrepareRequestDataPlugin) PrepareRequestData(ctx context.Context, request *schedulingtypes.LLMRequest, pods []schedulingtypes.Pod) error {
+	m.executed = true
+	if m.delay > 0 {
+		select {
+		case <-time.After(m.delay):
+		case <-ctx.Done():
+			return ctx.Err()
+		}
+	}
+	return m.returnErr
+}
+
+func (m *mockPrepareRequestDataPlugin) Produces() map[string]any {
+	return nil
+}
+
+func (m *mockPrepareRequestDataPlugin) Consumes() map[string]any {
+	return nil
+}
+
+func TestPrepareDataPluginsWithTimeout(t *testing.T) {
+	testCases := []struct {
+		name          string
+		timeout       time.Duration
+		plugins       []PrepareDataPlugin
+		ctxFn         func() (context.Context, context.CancelFunc)
+		expectErrStr  string
+		checkPlugins  func(t *testing.T, plugins []PrepareDataPlugin)
+		expectSuccess bool
+	}{
+		{
+			name:    "success with one plugin",
+			timeout: 100 * time.Millisecond,
+			plugins: []PrepareDataPlugin{
+				&mockPrepareRequestDataPlugin{name: "p1"},
+			},
+			ctxFn: func() (context.Context, context.CancelFunc) {
+				return context.Background(), func() {}
+			},
+			expectSuccess: true,
+			checkPlugins: func(t *testing.T, plugins []PrepareDataPlugin) {
+				assert.True(t, plugins[0].(*mockPrepareRequestDataPlugin).executed)
+			},
+		},
+		{
+			name:    "plugin returns error",
+			timeout: 100 * time.Millisecond,
+			plugins: []PrepareDataPlugin{
+				&mockPrepareRequestDataPlugin{name: "p1", returnErr: errors.New("plugin failed")},
+			},
+			ctxFn: func() (context.Context, context.CancelFunc) {
+				return context.Background(), func() {}
+			},
+			expectErrStr: "prepare data plugin p1/mock failed: plugin failed",
+		},
+		{
+			name:    "plugins time out",
+			timeout: 50 * time.Millisecond,
+			plugins: []PrepareDataPlugin{
+				&mockPrepareRequestDataPlugin{name: "p1", delay: 100 * time.Millisecond},
+			},
+			ctxFn: func() (context.Context, context.CancelFunc) {
+				return context.Background(), func() {}
+			},
+			expectErrStr: "prepare data plugin timed out",
+		},
+		{
+			name:    "context cancelled",
+			timeout: 200 * time.Millisecond,
+			plugins: []PrepareDataPlugin{
+				&mockPrepareRequestDataPlugin{name: "p1", delay: 100 * time.Millisecond},
+			},
+			ctxFn: func() (context.Context, context.CancelFunc) {
+				ctx, cancel := context.WithCancel(context.Background())
+				time.AfterFunc(50*time.Millisecond, cancel)
+				return ctx, cancel
+			},
+			expectErrStr: "context canceled",
+		},
+		{
+			name:    "multiple plugins success",
+			timeout: 100 * time.Millisecond,
+			plugins: []PrepareDataPlugin{
+				&mockPrepareRequestDataPlugin{name: "p1"},
+				&mockPrepareRequestDataPlugin{name: "p2"},
+			},
+			ctxFn: func() (context.Context, context.CancelFunc) {
+				return context.Background(), func() {}
+			},
+			expectSuccess: true,
+			checkPlugins: func(t *testing.T, plugins []PrepareDataPlugin) {
+				assert.True(t, plugins[0].(*mockPrepareRequestDataPlugin).executed)
+				assert.True(t, plugins[1].(*mockPrepareRequestDataPlugin).executed)
+			},
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			ctx, cancel := tc.ctxFn()
+			defer cancel()
+
+			err := prepareDataPluginsWithTimeout(tc.timeout, tc.plugins, ctx, &schedulingtypes.LLMRequest{}, nil)
+
+			if tc.expectSuccess {
+				assert.NoError(t, err)
+			} else {
+				assert.Error(t, err)
+				assert.Contains(t, err.Error(), tc.expectErrStr)
+			}
+
+			if tc.checkPlugins != nil {
+				tc.checkPlugins(t, tc.plugins)
+			}
+		})
+	}
+}
diff --git a/pkg/epp/saturationdetector/saturationdetector_test.go b/pkg/epp/saturationdetector/saturationdetector_test.go
index 0b861d9..833d0b2 100644
--- a/pkg/epp/saturationdetector/saturationdetector_test.go
+++ b/pkg/epp/saturationdetector/saturationdetector_test.go
@@ -18,14 +18,10 @@ package saturationdetector
 
 import (
 	"context"
-	"fmt"
-	"os"
-	"strconv"
 	"testing"
 	"time"
 
 	"github.com/go-logr/logr"
-	"github.com/google/go-cmp/cmp"
 	"k8s.io/apimachinery/pkg/types"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
@@ -43,68 +39,6 @@ func newMockPodMetrics(name string, metrics *backendmetrics.MetricsState) *backe
 
 // --- Tests ---
 
-func TestNewDetector(t *testing.T) {
-	tests := []struct {
-		name           string
-		config         *Config
-		expectedConfig *Config
-	}{
-		{
-			name: "Valid config",
-			config: &Config{
-				QueueDepthThreshold:       10,
-				KVCacheUtilThreshold:      0.8,
-				MetricsStalenessThreshold: 100 * time.Millisecond,
-			},
-			expectedConfig: &Config{
-				QueueDepthThreshold:       10,
-				KVCacheUtilThreshold:      0.8,
-				MetricsStalenessThreshold: 100 * time.Millisecond,
-			},
-		},
-		{
-			name: "invalid thresholds, fallback to default",
-			config: &Config{
-				QueueDepthThreshold:       -1,
-				KVCacheUtilThreshold:      -5,
-				MetricsStalenessThreshold: 0,
-			},
-			expectedConfig: &Config{
-				QueueDepthThreshold:       DefaultQueueDepthThreshold,
-				KVCacheUtilThreshold:      DefaultKVCacheUtilThreshold,
-				MetricsStalenessThreshold: DefaultMetricsStalenessThreshold,
-			},
-		},
-		{
-			name: "kv cache threshold above range, fallback to default",
-			config: &Config{
-				QueueDepthThreshold:       10,
-				KVCacheUtilThreshold:      1.5,
-				MetricsStalenessThreshold: 100 * time.Millisecond,
-			},
-			expectedConfig: &Config{
-				QueueDepthThreshold:       10,
-				KVCacheUtilThreshold:      DefaultKVCacheUtilThreshold,
-				MetricsStalenessThreshold: 100 * time.Millisecond,
-			},
-		},
-	}
-
-	for _, test := range tests {
-		t.Run(test.name, func(t *testing.T) {
-			// validate configuration values are loaded from env vars properly, including the use of default values when provided value is invalid.
-			os.Setenv(EnvSdQueueDepthThreshold, strconv.Itoa(test.config.QueueDepthThreshold))
-			os.Setenv(EnvSdKVCacheUtilThreshold, fmt.Sprintf("%v", test.config.KVCacheUtilThreshold))
-			os.Setenv(EnvSdMetricsStalenessThreshold, test.config.MetricsStalenessThreshold.String())
-
-			detector := NewDetector(LoadConfigFromEnv(), logr.Discard())
-			if diff := cmp.Diff(test.expectedConfig, detector.config); diff != "" {
-				t.Errorf("Unexpected output (-want +got): %v", diff)
-			}
-		})
-	}
-}
-
 func TestDetector_IsSaturated(t *testing.T) {
 	baseTime := time.Now()
 	defaultConfig := &Config{
diff --git a/pkg/epp/scheduling/framework/plugins/test/consts.go b/pkg/epp/scheduling/framework/plugins/test/consts.go
index 81c0579..166dba2 100644
--- a/pkg/epp/scheduling/framework/plugins/test/consts.go
+++ b/pkg/epp/scheduling/framework/plugins/test/consts.go
@@ -17,9 +17,22 @@ limitations under the License.
 package test
 
 const (
-	// HeaderTestEppEndPointSelectionKey is the header used for testing purposes to make EPP behavior controllable.
-	// The header value should be a comma-separated list of endpoint IP addresses.
-	// E.g., "test-epp-endpoint-selection": "10.0.0.7,10.0.0.8"
-	// The returned order is the same as the order provided in the header.
+	// HeaderTestEppEndPointSelectionKey is the request header used in tests to control
+	// Endpoint Picker (EPP) behavior deterministically.
+	//
+	// The header value is a comma-separated list of endpoint identifiers. Each entry
+	// may be in one of the following formats:
+	//
+	//   - "IP"          — selects all pods whose IP address matches the given value.
+	//   - "IP:port"     — selects only pods whose IP and port both match exactly.
+	//                     Ports correspond to data-parallel ranks or specific targetPorts.
+	//
+	// IPv6 addresses are supported, with or without brackets (e.g. "fd00::1" or "[fd00::1]:3002").
+	// The returned order matches the order of endpoints specified in the header, and duplicates
+	// are ignored.
+	//
+	// Examples:
+	//   "test-epp-endpoint-selection": "10.0.0.7,10.0.0.8:3002"
+	//   "test-epp-endpoint-selection": "[fd00::1]:3000,fd00::2"
 	HeaderTestEppEndPointSelectionKey = "test-epp-endpoint-selection"
 )
diff --git a/pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go b/pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go
index d5615bc..3cd8a61 100644
--- a/pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go
+++ b/pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go
@@ -35,88 +35,93 @@ func TestFilter(t *testing.T) {
 		output []types.Pod
 	}{
 		{
-			name: "TestHeaderBasedFilter, header endpoint unset in request",
-			req:  &types.LLMRequest{}, // Delieverately unset the header.
+			name: "header unset in request",
+			req:  &types.LLMRequest{}, // Deliberately unset
 			input: []types.Pod{
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint",
-					},
-				},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3000"}},
 			},
 			output: []types.Pod{},
 		},
 		{
-			name: "TestHeaderBasedFilter, header endpoint set in request but no match",
-			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "test-endpoint"}},
+			name: "header set but no IP match",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "10.0.0.99"}},
 			input: []types.Pod{
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint-unmatch",
-					},
-				},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3000"}},
 			},
 			output: []types.Pod{},
 		},
 		{
-			name: "TestHeaderBasedFilter, header endpoint set",
-			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "test-endpoint"}},
+			name: "IP-only header matches pod (port-agnostic)",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "10.0.0.1"}},
 			input: []types.Pod{
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint",
-					},
-				},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3002"}},
 			},
 			output: []types.Pod{
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint",
-					},
-				},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3002"}},
 			},
 		},
 		{
-			name: "TestHeaderBasedFilter, multiple header endpoints set and multiple matches",
-			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "test-endpoint3,test-endpoint2"}},
+			name: "IP:port header matches exact port",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "10.0.0.1:3002"}},
 			input: []types.Pod{
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint1",
-					},
-				},
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint2",
-					},
-				},
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint3",
-					},
-				},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3000"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3002"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.2", Port: "3002"}},
 			},
 			output: []types.Pod{
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint3",
-					},
-				},
-				&types.PodMetrics{
-					Pod: &backend.Pod{
-						Address: "test-endpoint2",
-					},
-				},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3002"}},
+			},
+		},
+		{
+			name: "IP:port header with non-matching port produces no match",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "10.0.0.1:9999"}},
+			input: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3002"}},
+			},
+			output: []types.Pod{},
+		},
+		{
+			name: "multiple header values (IP and IP:port) produce multiple matches in order and deduped",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "10.0.0.3:3004, 10.0.0.2, 10.0.0.3"}},
+			input: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.1", Port: "3000"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.2", Port: "3002"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.3", Port: "3004"}},
+			},
+			output: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.3", Port: "3004"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "10.0.0.2", Port: "3002"}},
+			},
+		},
+		{
+			name: "IPv6 with brackets and port",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "[fd00::1]:3002"}},
+			input: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "fd00::1", Port: "3002"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "fd00::2", Port: "3002"}},
+			},
+			output: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "fd00::1", Port: "3002"}},
+			},
+		},
+		{
+			name: "IPv6 bare address (no port)",
+			req:  &types.LLMRequest{Headers: map[string]string{test.HeaderTestEppEndPointSelectionKey: "fd00::2"}},
+			input: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "fd00::1", Port: "3002"}},
+				&types.PodMetrics{Pod: &backend.Pod{Address: "fd00::2", Port: "3004"}},
+			},
+			output: []types.Pod{
+				&types.PodMetrics{Pod: &backend.Pod{Address: "fd00::2", Port: "3004"}},
 			},
 		},
 	}
 
-	for _, test := range tests {
-		t.Run(test.name, func(t *testing.T) {
-			got := NewHeaderBasedTestingFilter().Filter(context.Background(), types.NewCycleState(), test.req, test.input)
-
-			if diff := cmp.Diff(test.output, got); diff != "" {
-				t.Errorf("Unexpected output (-want +got): %v", diff)
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			got := NewHeaderBasedTestingFilter().Filter(context.Background(), types.NewCycleState(), tc.req, tc.input)
+			if diff := cmp.Diff(tc.output, got); diff != "" {
+				t.Fatalf("Unexpected output (-want +got): %s", diff)
 			}
 		})
 	}
diff --git a/pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go b/pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go
index 2836755..bce3be1 100644
--- a/pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go
@@ -19,6 +19,7 @@ package filter
 import (
 	"context"
 	"encoding/json"
+	"net"
 	"strings"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
@@ -64,24 +65,74 @@ func (f *HeaderBasedTestingFilter) WithName(name string) *HeaderBasedTestingFilt
 	return f
 }
 
-// Filter selects pods that match the IP addresses specified in the request header.
+// Filter selects pods whose IP or IP:port matches any value in the
+// "test-epp-endpoint-selection" header. Values may be "IP" or "IP:port".
+// If a port is provided, only an exact IP:port match is accepted.
 func (f *HeaderBasedTestingFilter) Filter(_ context.Context, _ *types.CycleState, request *types.LLMRequest, pods []types.Pod) []types.Pod {
-	headerValue, ok := request.Headers[test.HeaderTestEppEndPointSelectionKey]
-	if !ok || headerValue == "" {
+	hv, ok := request.Headers[test.HeaderTestEppEndPointSelectionKey]
+	if !ok || strings.TrimSpace(hv) == "" {
 		return []types.Pod{}
 	}
 
-	podAddressMap := make(map[string]types.Pod, len(pods))
-	for _, pod := range pods {
-		podAddressMap[pod.GetPod().GetIPAddress()] = pod
+	normalizeIP := func(s string) string { return strings.Trim(s, "[]") }
+
+	// Build lookup maps:
+	//   ip -> pod
+	//   ip:port -> pod (only when pod GetPort() is non-empty)
+	ipToPod := make(map[string]types.Pod, len(pods))
+	hpToPod := make(map[string]types.Pod, len(pods))
+	for _, p := range pods {
+		if p == nil || p.GetPod() == nil {
+			continue
+		}
+		ip := normalizeIP(strings.TrimSpace(p.GetPod().GetIPAddress()))
+		if ip == "" {
+			continue
+		}
+		ipToPod[ip] = p
+		if port := strings.TrimSpace(p.GetPod().GetPort()); port != "" {
+			hpToPod[ip+":"+port] = p
+		}
 	}
 
-	endpoints := strings.Split(headerValue, ",")
-	filteredPods := make([]types.Pod, 0, len(endpoints))
-	for _, endpoint := range endpoints {
-		trimmedEndpoint := strings.TrimSpace(endpoint)
-		if pod, found := podAddressMap[trimmedEndpoint]; found {
-			filteredPods = append(filteredPods, pod)
+	headerVals := strings.Split(hv, ",")
+	filteredPods := make([]types.Pod, 0, len(headerVals))
+	seen := make(map[string]struct{}, len(headerVals)) // de-dupe by pod IP
+
+	for _, raw := range headerVals {
+		item := strings.TrimSpace(raw)
+		if item == "" {
+			continue
+		}
+
+		host := item
+		port := ""
+		if h, pt, err := net.SplitHostPort(item); err == nil {
+			host, port = h, pt
+		} else {
+			host = normalizeIP(host) // bare IP, possibly bracketed IPv6
+		}
+		host = normalizeIP(host)
+
+		var pod types.Pod
+		if port != "" {
+			// Require an exact ip:port match
+			if p, ok := hpToPod[host+":"+port]; ok {
+				pod = p
+			}
+		} else {
+			// IP-only selection
+			if p, ok := ipToPod[host]; ok {
+				pod = p
+			}
+		}
+
+		if pod != nil {
+			ip := normalizeIP(pod.GetPod().GetIPAddress())
+			if _, dup := seen[ip]; !dup {
+				seen[ip] = struct{}{}
+				filteredPods = append(filteredPods, pod)
+			}
 		}
 	}
 	return filteredPods
diff --git a/test/e2e/epp/e2e_suite_test.go b/test/e2e/epp/e2e_suite_test.go
index b41f108..504e59e 100644
--- a/test/e2e/epp/e2e_suite_test.go
+++ b/test/e2e/epp/e2e_suite_test.go
@@ -103,7 +103,7 @@ var _ = ginkgo.BeforeSuite(func() {
 	if nsName == "" {
 		nsName = defaultNsName
 	}
-	testConfig = testutils.NewTestConfig(nsName)
+	testConfig = testutils.NewTestConfig(nsName, "")
 
 	e2eImage = os.Getenv("E2E_IMAGE")
 	gomega.Expect(e2eImage).NotTo(gomega.BeEmpty(), "E2E_IMAGE environment variable is not set")
diff --git a/test/testdata/configloader_1_test.yaml b/test/testdata/configloader_1_test.yaml
index db75a42..f436841 100644
--- a/test/testdata/configloader_1_test.yaml
+++ b/test/testdata/configloader_1_test.yaml
@@ -12,7 +12,6 @@ plugins:
     blockSize: 32
 - name: testPicker
   type: test-picker
-
 schedulingProfiles:
 - name: default
   plugins:
@@ -20,3 +19,7 @@ schedulingProfiles:
   - pluginRef: test-two
     weight: 50
   - pluginRef: testPicker
+featureGates:
+- dataLayer
+saturationDetector:
+  metricsStalenessThreshold: 150ms
diff --git a/test/utils/utils.go b/test/utils/utils.go
index b65834c..c2d9769 100644
--- a/test/utils/utils.go
+++ b/test/utils/utils.go
@@ -73,8 +73,9 @@ type TestConfig struct {
 }
 
 // NewTestConfig creates a new TestConfig instance
-func NewTestConfig(nsName string) *TestConfig {
-	cfg := config.GetConfigOrDie()
+func NewTestConfig(nsName string, k8sContext string) *TestConfig {
+	cfg, err := config.GetConfigWithContext(k8sContext)
+	gomega.Expect(err).NotTo(gomega.HaveOccurred())
 	gomega.Expect(cfg).NotTo(gomega.BeNil())
 
 	kubeCli, err := kubernetes.NewForConfig(cfg)
@@ -458,11 +459,11 @@ func DeleteObjects(testConfig *TestConfig, kindAndNames []string) {
 	}
 }
 
-// applyYAMLFile reads a file containing YAML (possibly multiple docs)
+// ApplyYAMLFile reads a file containing YAML (possibly multiple docs)
 // and applies each object to the cluster.
-func ApplyYAMLFile(testConfig *TestConfig, filePath string) {
+func ApplyYAMLFile(testConfig *TestConfig, filePath string) []string {
 	// Create the resources from the manifest file
-	CreateObjsFromYaml(testConfig, ReadYaml(filePath))
+	return CreateObjsFromYaml(testConfig, ReadYaml(filePath))
 }
 
 // ReadYaml is a helper function to read in K8S YAML files and split by the --- separator