Repository: kubernetes-sigs/gateway-api-inference-extension

Issue:
remove dependency of code under cmd on conformance
as of today, the code under cmd dir and more specifically inside runner has reference to conformance package.
this includes also the need to copy conformance dir in the Dockerfile.

the only reason for this dependency is the fact that conformance tests need to have a predictable output and therefore a new filter plugin was introduced in the conformance tests. 
this is initialized using env var that is set to `True` and then the scheduler is initialized with a single profile that includes a single (predictable) filter.

see the relevant code here:
https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/b4d1e674fe5d0702d20be96469c12709e2f3a8fc/cmd/epp/runner/runner.go#L324-L325

I think code under cmd shouldn't have any dependency on conformance. the other direction is fine though.
I would like to suggest an alternative approach to remove this dependency (and remove all relevant code like the env var). the suggested change will keep all conformance code encapsulated inside conformance dir, without leaking outside to any other package.

recently some changes were made to epp cmd to split the configuration from `main.go` and into runner package.
these changes result in a very thin `main.go`, which can configure plugins via code.

we could have under conformance package this kind of `main.go` file (e.g., under `conformance/cmd`) and a Dockerfile that builds an image we may call `epp-conformance`.
the `main.go` file would look something like the following:
```go
func main() {
	if err := runner.NewRunner().
		WithSchedulerConfig(schedulerConfig). // conformance config, includes a single profile with a predictable filter
		Run(ctrl.SetupSignalHandler()); err != nil {
		os.Exit(1)
	}
}
```
obviously, the image that will be built is completely identical to the regular epp, with one difference that we set in `main.go` the predictable scheduler config.
then we could remove conformance dependency from the regular main.go under epp/cmd and from the Dockerfile.

the main advantage is It's a clean solution for removing the conformance dependency.
the downside is that we need to maintain additional docker image (which should be built automatically, same as epp image is built after every PR merges). However I believe it's justified in this case.

I discussed this with @SinaChavoshi shortly in slack, and he sounds supportive.

I'd be happy to hear more thoughts @kfswain @danehans @robscott @ahg-g @SinaChavoshi  

Discussion:
@ahg-g: Shouldn't we be able to use a config file to setup the necessary plugins for conformance tests?@nirrozenbaum: yes, when ready. 
we might need main anyway to register the filter@ahg-g: We are close to having it ready, and so I don't see a strong reason to create another main.go and dockerfile when we can do the same thing via a config file on the released image.

@ahg-g: > we might need main anyway to register the filter

I think we should just ship this filter with the released epp, I don't see a reason not to.@nirrozenbaum: if we ship it with released epp then no need for additional image. I’m cool with that. just want to make sure we remove the dependency on conformance

Solution patch:
diff --git a/Dockerfile b/Dockerfile
index 5512518..fb73765 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -19,7 +19,6 @@ RUN go mod download
 # Sources
 COPY cmd/epp ./cmd/epp
 COPY pkg/epp ./pkg/epp
-COPY conformance/testing-epp ./conformance/testing-epp
 COPY internal ./internal
 COPY api ./api
 WORKDIR /src/cmd/epp
diff --git a/api/v1alpha2/inferencepool_types.go b/api/v1alpha2/inferencepool_types.go
index 90228f6..1edb7ce 100644
--- a/api/v1alpha2/inferencepool_types.go
+++ b/api/v1alpha2/inferencepool_types.go
@@ -17,7 +17,6 @@ limitations under the License.
 package v1alpha2
 
 import (
-	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
@@ -182,7 +181,7 @@ type InferencePoolStatus struct {
 // PoolStatus defines the observed state of InferencePool from a Gateway.
 type PoolStatus struct {
 	// GatewayRef indicates the gateway that observed state of InferencePool.
-	GatewayRef corev1.ObjectReference `json:"parentRef"`
+	GatewayRef ParentGatewayReference `json:"parentRef"`
 
 	// Conditions track the state of the InferencePool.
 	//
@@ -270,3 +269,29 @@ const (
 	// or API group, or a reference to a resource that can not be found.
 	InferencePoolReasonInvalidExtensionRef InferencePoolReason = "InvalidExtensionRef"
 )
+
+// ParentGatewayReference identifies an API object including its namespace,
+// defaulting to Gateway.
+type ParentGatewayReference struct {
+	// Group is the group of the referent.
+	//
+	// +optional
+	// +kubebuilder:default="gateway.networking.k8s.io"
+	Group *Group `json:"group"`
+
+	// Kind is kind of the referent. For example "Gateway".
+	//
+	// +optional
+	// +kubebuilder:default=Gateway
+	Kind *Kind `json:"kind"`
+
+	// Name is the name of the referent.
+	Name ObjectName `json:"name"`
+
+	// Namespace is the namespace of the referent.  If not present,
+	// the namespace of the referent is assumed to be the same as
+	// the namespace of the referring object.
+	//
+	// +optional
+	Namespace *Namespace `json:"namespace,omitempty"`
+}
diff --git a/api/v1alpha2/shared_types.go b/api/v1alpha2/shared_types.go
index ea5ef29..d3829ef 100644
--- a/api/v1alpha2/shared_types.go
+++ b/api/v1alpha2/shared_types.go
@@ -106,3 +106,24 @@ type LabelKey string
 // +kubebuilder:validation:MaxLength=63
 // +kubebuilder:validation:Pattern=`^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?$`
 type LabelValue string
+
+// Namespace refers to a Kubernetes namespace. It must be a RFC 1123 label.
+//
+// This validation is based off of the corresponding Kubernetes validation:
+// https://github.com/kubernetes/apimachinery/blob/02cfb53916346d085a6c6c7c66f882e3c6b0eca6/pkg/util/validation/validation.go#L187
+//
+// This is used for Namespace name validation here:
+// https://github.com/kubernetes/apimachinery/blob/02cfb53916346d085a6c6c7c66f882e3c6b0eca6/pkg/api/validation/generic.go#L63
+//
+// Valid values include:
+//
+// * "example"
+//
+// Invalid values include:
+//
+// * "example.com" - "." is an invalid character
+//
+// +kubebuilder:validation:Pattern=`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`
+// +kubebuilder:validation:MinLength=1
+// +kubebuilder:validation:MaxLength=63
+type Namespace string
diff --git a/api/v1alpha2/zz_generated.deepcopy.go b/api/v1alpha2/zz_generated.deepcopy.go
index 218df16..f064480 100644
--- a/api/v1alpha2/zz_generated.deepcopy.go
+++ b/api/v1alpha2/zz_generated.deepcopy.go
@@ -325,6 +325,36 @@ func (in *InferencePoolStatus) DeepCopy() *InferencePoolStatus {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *ParentGatewayReference) DeepCopyInto(out *ParentGatewayReference) {
+	*out = *in
+	if in.Group != nil {
+		in, out := &in.Group, &out.Group
+		*out = new(Group)
+		**out = **in
+	}
+	if in.Kind != nil {
+		in, out := &in.Kind, &out.Kind
+		*out = new(Kind)
+		**out = **in
+	}
+	if in.Namespace != nil {
+		in, out := &in.Namespace, &out.Namespace
+		*out = new(Namespace)
+		**out = **in
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ParentGatewayReference.
+func (in *ParentGatewayReference) DeepCopy() *ParentGatewayReference {
+	if in == nil {
+		return nil
+	}
+	out := new(ParentGatewayReference)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *PoolObjectReference) DeepCopyInto(out *PoolObjectReference) {
 	*out = *in
@@ -343,7 +373,7 @@ func (in *PoolObjectReference) DeepCopy() *PoolObjectReference {
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *PoolStatus) DeepCopyInto(out *PoolStatus) {
 	*out = *in
-	out.GatewayRef = in.GatewayRef
+	in.GatewayRef.DeepCopyInto(&out.GatewayRef)
 	if in.Conditions != nil {
 		in, out := &in.Conditions, &out.Conditions
 		*out = make([]v1.Condition, len(*in))
diff --git a/client-go/applyconfiguration/api/v1alpha2/parentgatewayreference.go b/client-go/applyconfiguration/api/v1alpha2/parentgatewayreference.go
new file mode 100644
index 0000000..a003b8a
--- /dev/null
+++ b/client-go/applyconfiguration/api/v1alpha2/parentgatewayreference.go
@@ -0,0 +1,70 @@
+/*
+Copyright The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package v1alpha2
+
+import (
+	apiv1alpha2 "sigs.k8s.io/gateway-api-inference-extension/api/v1alpha2"
+)
+
+// ParentGatewayReferenceApplyConfiguration represents a declarative configuration of the ParentGatewayReference type for use
+// with apply.
+type ParentGatewayReferenceApplyConfiguration struct {
+	Group     *apiv1alpha2.Group      `json:"group,omitempty"`
+	Kind      *apiv1alpha2.Kind       `json:"kind,omitempty"`
+	Name      *apiv1alpha2.ObjectName `json:"name,omitempty"`
+	Namespace *apiv1alpha2.Namespace  `json:"namespace,omitempty"`
+}
+
+// ParentGatewayReferenceApplyConfiguration constructs a declarative configuration of the ParentGatewayReference type for use with
+// apply.
+func ParentGatewayReference() *ParentGatewayReferenceApplyConfiguration {
+	return &ParentGatewayReferenceApplyConfiguration{}
+}
+
+// WithGroup sets the Group field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Group field is set to the value of the last call.
+func (b *ParentGatewayReferenceApplyConfiguration) WithGroup(value apiv1alpha2.Group) *ParentGatewayReferenceApplyConfiguration {
+	b.Group = &value
+	return b
+}
+
+// WithKind sets the Kind field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Kind field is set to the value of the last call.
+func (b *ParentGatewayReferenceApplyConfiguration) WithKind(value apiv1alpha2.Kind) *ParentGatewayReferenceApplyConfiguration {
+	b.Kind = &value
+	return b
+}
+
+// WithName sets the Name field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Name field is set to the value of the last call.
+func (b *ParentGatewayReferenceApplyConfiguration) WithName(value apiv1alpha2.ObjectName) *ParentGatewayReferenceApplyConfiguration {
+	b.Name = &value
+	return b
+}
+
+// WithNamespace sets the Namespace field in the declarative configuration to the given value
+// and returns the receiver, so that objects can be built by chaining "With" function invocations.
+// If called multiple times, the Namespace field is set to the value of the last call.
+func (b *ParentGatewayReferenceApplyConfiguration) WithNamespace(value apiv1alpha2.Namespace) *ParentGatewayReferenceApplyConfiguration {
+	b.Namespace = &value
+	return b
+}
diff --git a/client-go/applyconfiguration/api/v1alpha2/poolstatus.go b/client-go/applyconfiguration/api/v1alpha2/poolstatus.go
index cfa651e..fcc0ddf 100644
--- a/client-go/applyconfiguration/api/v1alpha2/poolstatus.go
+++ b/client-go/applyconfiguration/api/v1alpha2/poolstatus.go
@@ -19,15 +19,14 @@ limitations under the License.
 package v1alpha2
 
 import (
-	v1 "k8s.io/api/core/v1"
-	metav1 "k8s.io/client-go/applyconfigurations/meta/v1"
+	v1 "k8s.io/client-go/applyconfigurations/meta/v1"
 )
 
 // PoolStatusApplyConfiguration represents a declarative configuration of the PoolStatus type for use
 // with apply.
 type PoolStatusApplyConfiguration struct {
-	GatewayRef *v1.ObjectReference                  `json:"parentRef,omitempty"`
-	Conditions []metav1.ConditionApplyConfiguration `json:"conditions,omitempty"`
+	GatewayRef *ParentGatewayReferenceApplyConfiguration `json:"parentRef,omitempty"`
+	Conditions []v1.ConditionApplyConfiguration          `json:"conditions,omitempty"`
 }
 
 // PoolStatusApplyConfiguration constructs a declarative configuration of the PoolStatus type for use with
@@ -39,15 +38,15 @@ func PoolStatus() *PoolStatusApplyConfiguration {
 // WithGatewayRef sets the GatewayRef field in the declarative configuration to the given value
 // and returns the receiver, so that objects can be built by chaining "With" function invocations.
 // If called multiple times, the GatewayRef field is set to the value of the last call.
-func (b *PoolStatusApplyConfiguration) WithGatewayRef(value v1.ObjectReference) *PoolStatusApplyConfiguration {
-	b.GatewayRef = &value
+func (b *PoolStatusApplyConfiguration) WithGatewayRef(value *ParentGatewayReferenceApplyConfiguration) *PoolStatusApplyConfiguration {
+	b.GatewayRef = value
 	return b
 }
 
 // WithConditions adds the given value to the Conditions field in the declarative configuration
 // and returns the receiver, so that objects can be build by chaining "With" function invocations.
 // If called multiple times, values provided by each call will be appended to the Conditions field.
-func (b *PoolStatusApplyConfiguration) WithConditions(values ...*metav1.ConditionApplyConfiguration) *PoolStatusApplyConfiguration {
+func (b *PoolStatusApplyConfiguration) WithConditions(values ...*v1.ConditionApplyConfiguration) *PoolStatusApplyConfiguration {
 	for i := range values {
 		if values[i] == nil {
 			panic("nil value passed to WithConditions")
diff --git a/client-go/applyconfiguration/utils.go b/client-go/applyconfiguration/utils.go
index eafadbe..ea204cd 100644
--- a/client-go/applyconfiguration/utils.go
+++ b/client-go/applyconfiguration/utils.go
@@ -52,6 +52,8 @@ func ForKind(kind schema.GroupVersionKind) interface{} {
 		return &apiv1alpha2.InferencePoolSpecApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("InferencePoolStatus"):
 		return &apiv1alpha2.InferencePoolStatusApplyConfiguration{}
+	case v1alpha2.SchemeGroupVersion.WithKind("ParentGatewayReference"):
+		return &apiv1alpha2.ParentGatewayReferenceApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("PoolObjectReference"):
 		return &apiv1alpha2.PoolObjectReferenceApplyConfiguration{}
 	case v1alpha2.SchemeGroupVersion.WithKind("PoolStatus"):
diff --git a/cmd/epp/runner/register.go b/cmd/epp/runner/register.go
index ce0bc35..161a224 100644
--- a/cmd/epp/runner/register.go
+++ b/cmd/epp/runner/register.go
@@ -25,6 +25,7 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/scorer"
+	testfilter "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/test/filter"
 )
 
 // RegisterAllPlugins registers the factory functions of all known plugins
@@ -40,6 +41,8 @@ func RegisterAllPlugins() {
 	plugins.Register(profile.SingleProfileHandlerType, profile.SingleProfileHandlerFactory)
 	plugins.Register(scorer.KvCacheScorerType, scorer.KvCacheScorerFactory)
 	plugins.Register(scorer.QueueScorerType, scorer.QueueScorerFactory)
+	// register filter for test purpose only (used in conformance tests)
+	plugins.Register(testfilter.HeaderBasedTestingFilterType, testfilter.HeaderBasedTestingFilterFactory)
 }
 
 // eppHandle is an implementation of the interface plugins.Handle
diff --git a/cmd/epp/runner/runner.go b/cmd/epp/runner/runner.go
index 551db31..28120dd 100644
--- a/cmd/epp/runner/runner.go
+++ b/cmd/epp/runner/runner.go
@@ -37,7 +37,6 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/metrics/filters"
 	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
 
-	conformance_epp "sigs.k8s.io/gateway-api-inference-extension/conformance/testing-epp"
 	"sigs.k8s.io/gateway-api-inference-extension/internal/runnable"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config/loader"
@@ -48,7 +47,6 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
 	runserver "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/server"
-	envutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/env"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
 )
 
@@ -65,6 +63,10 @@ var (
 		"metricsPort",
 		runserver.DefaultMetricsPort,
 		"The metrics port")
+	enablePprof = flag.Bool(
+		"enablePprof",
+		runserver.DefaultEnablePprof,
+		"Enables pprof handlers. Defaults to true. Set to false to disable pprof handlers.")
 	destinationEndpointHintKey = flag.String(
 		"destinationEndpointHintKey",
 		runserver.DefaultDestinationEndpointHintKey,
@@ -137,9 +139,6 @@ var (
 	modelServerMetricsPath = flag.String("modelServerMetricsPath", "/metrics", "Path to scrape metrics from pods")
 
 	setupLog = ctrl.Log.WithName("setup")
-
-	// Environment variables
-	reqHeaderBasedSchedulerForTesting = envutil.GetEnvBool("ENABLE_REQ_HEADER_BASED_SCHEDULER_FOR_TESTING", false, setupLog)
 )
 
 // NewRunner initializes a new EPP Runner and returns its pointer.
@@ -263,10 +262,14 @@ func (r *Runner) Run(ctx context.Context) error {
 		setupLog.Error(err, "Failed to create controller manager")
 		return err
 	}
-	err = setupPprofHandlers(mgr)
-	if err != nil {
-		setupLog.Error(err, "Failed to setup pprof handlers")
-		return err
+
+	if *enablePprof {
+		setupLog.Info("Enabling pprof handlers")
+		err = setupPprofHandlers(mgr)
+		if err != nil {
+			setupLog.Error(err, "Failed to setup pprof handlers")
+			return err
+		}
 	}
 
 	err = r.parseConfiguration(ctx)
@@ -329,13 +332,7 @@ func (r *Runner) initializeScheduler() *scheduling.Scheduler {
 	}
 
 	// otherwise, no one configured from outside scheduler config. use existing configuration
-	scheduler := scheduling.NewScheduler()
-
-	if reqHeaderBasedSchedulerForTesting {
-		scheduler = conformance_epp.NewReqHeaderBasedScheduler()
-	}
-
-	return scheduler
+	return scheduling.NewScheduler()
 }
 
 func (r *Runner) parseConfiguration(ctx context.Context) error {
diff --git a/config/charts/inferencepool/templates/epp-deployment.yaml b/config/charts/inferencepool/templates/epp-deployment.yaml
index 0fcab49..fec91e4 100644
--- a/config/charts/inferencepool/templates/epp-deployment.yaml
+++ b/config/charts/inferencepool/templates/epp-deployment.yaml
@@ -35,6 +35,8 @@ spec:
         - "9003"
         - -metricsPort
         - "9090"
+        # https://pkg.go.dev/flag#hdr-Command_line_flag_syntax; space is only for non-bool flags
+        - "-enablePprof={{ .Values.inferenceExtension.enablePprof }}"
         {{- if eq (.Values.inferencePool.modelServerType | default "vllm") "triton-tensorrt-llm" }}
         - -totalQueuedRequestsMetric
         - "nv_trt_llm_request_metrics{request_type=waiting}"
diff --git a/config/charts/inferencepool/values.yaml b/config/charts/inferencepool/values.yaml
index f793d86..2b4e800 100644
--- a/config/charts/inferencepool/values.yaml
+++ b/config/charts/inferencepool/values.yaml
@@ -7,6 +7,7 @@ inferenceExtension:
     pullPolicy: Always
   extProcPort: 9002
   env: {}
+  enablePprof: true # Enable pprof handlers for profiling and debugging
   # Example environment variables:
   # env:
   #   KV_CACHE_SCORE_WEIGHT: "1"
diff --git a/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml b/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
index db342d0..0c5f1b7 100644
--- a/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
+++ b/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
@@ -242,46 +242,36 @@ spec:
                       description: GatewayRef indicates the gateway that observed
                         state of InferencePool.
                       properties:
-                        apiVersion:
-                          description: API version of the referent.
-                          type: string
-                        fieldPath:
-                          description: |-
-                            If referring to a piece of an object instead of an entire object, this string
-                            should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
-                            For example, if the object reference is to a container within a pod, this would take on a value like:
-                            "spec.containers{name}" (where "name" refers to the name of the container that triggered
-                            the event) or if no container name is specified "spec.containers[2]" (container with
-                            index 2 in this pod). This syntax is chosen only to have some well-defined way of
-                            referencing a part of an object.
+                        group:
+                          default: gateway.networking.k8s.io
+                          description: Group is the group of the referent.
+                          maxLength: 253
+                          pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
                           type: string
                         kind:
-                          description: |-
-                            Kind of the referent.
-                            More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
+                          default: Gateway
+                          description: Kind is kind of the referent. For example "Gateway".
+                          maxLength: 63
+                          minLength: 1
+                          pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
                           type: string
                         name:
-                          description: |-
-                            Name of the referent.
-                            More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+                          description: Name is the name of the referent.
+                          maxLength: 253
+                          minLength: 1
                           type: string
                         namespace:
                           description: |-
-                            Namespace of the referent.
-                            More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
-                          type: string
-                        resourceVersion:
-                          description: |-
-                            Specific resourceVersion to which this reference is made, if any.
-                            More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
-                          type: string
-                        uid:
-                          description: |-
-                            UID of the referent.
-                            More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
+                            Namespace is the namespace of the referent.  If not present,
+                            the namespace of the referent is assumed to be the same as
+                            the namespace of the referring object.
+                          maxLength: 63
+                          minLength: 1
+                          pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
                           type: string
+                      required:
+                      - name
                       type: object
-                      x-kubernetes-map-type: atomic
                   required:
                   - parentRef
                   type: object
diff --git a/conformance/resources/manifests/manifests.yaml b/conformance/resources/manifests/manifests.yaml
index 5b1f2d4..5fbcfdc 100644
--- a/conformance/resources/manifests/manifests.yaml
+++ b/conformance/resources/manifests/manifests.yaml
@@ -211,11 +211,11 @@ spec:
         - "9002"
         - -grpcHealthPort
         - "9003"
+        - "-configFile"
+        - "/config/conformance-plugins.yaml"
         env:
         - name: USE_STREAMING
           value: "true"
-        - name: ENABLE_REQ_HEADER_BASED_SCHEDULER_FOR_TESTING # Used for conformance test.
-          value: "true"
         ports:
         - containerPort: 9002
         - containerPort: 9003
@@ -233,6 +233,13 @@ spec:
             service: inference-extension
           initialDelaySeconds: 5
           periodSeconds: 10
+        volumeMounts:
+        - name: plugins-config-volume
+          mountPath: "/config"
+      volumes:
+      - name: plugins-config-volume
+        configMap:
+          name: plugins-config
 ---
 # --- Secondary InferencePool Definition ---
 apiVersion: inference.networking.x-k8s.io/v1alpha2
@@ -301,11 +308,11 @@ spec:
         - "9002"
         - -grpcHealthPort
         - "9003"
+        - "-configFile"
+        - "/config/conformance-plugins.yaml"
         env:
         - name: USE_STREAMING
           value: "true"
-        - name: ENABLE_REQ_HEADER_BASED_SCHEDULER_FOR_TESTING # Used for conformance test.
-          value: "true"
         ports:
         - containerPort: 9002
         - containerPort: 9003
@@ -323,6 +330,34 @@ spec:
             service: inference-extension
           initialDelaySeconds: 5
           periodSeconds: 10
+        volumeMounts:
+        - name: plugins-config-volume
+          mountPath: "/config"
+      volumes:
+      - name: plugins-config-volume
+        configMap:
+          name: plugins-config
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: plugins-config
+  namespace: default
+data:
+  conformance-plugins.yaml: |
+    apiVersion: inference.networking.x-k8s.io/v1alpha1
+    kind: EndpointPickerConfig
+    plugins:
+    - type: header-based-testing-filter
+    - type: max-score-picker
+      parameters:
+        maxNumOfEndpoints: 1
+    - type: single-profile-handler
+    schedulingProfiles:
+    - name: conformance-profile
+      plugins:
+      - pluginRef: header-based-testing-filter
+      - pluginRef: max-score-picker
 ---
 # --- Required Role and RoleBinding for Conformance Test for EPP ---
 apiVersion: rbac.authorization.k8s.io/v1
diff --git a/conformance/utils/kubernetes/helpers.go b/conformance/utils/kubernetes/helpers.go
index f2e1e21..6f40999 100644
--- a/conformance/utils/kubernetes/helpers.go
+++ b/conformance/utils/kubernetes/helpers.go
@@ -127,7 +127,8 @@ func InferencePoolMustHaveCondition(t *testing.T, c client.Reader, poolNN types.
 				debugMsg += " (No parent statuses reported)"
 			}
 			for i, parentStatus := range lastObservedPool.Status.Parents {
-				debugMsg += fmt.Sprintf("\n  Parent %d (Gateway: %s/%s):", i, parentStatus.GatewayRef.Namespace, parentStatus.GatewayRef.Name)
+				namespace := parentStatus.GatewayRef.Namespace
+				debugMsg += fmt.Sprintf("\n  Parent %d (Gateway: %s/%s):", i, *namespace, parentStatus.GatewayRef.Name)
 				if len(parentStatus.Conditions) == 0 {
 					debugMsg += " (No conditions reported for this parent)"
 				}
diff --git a/pkg/epp/server/runserver.go b/pkg/epp/server/runserver.go
index 67dc78e..10da323 100644
--- a/pkg/epp/server/runserver.go
+++ b/pkg/epp/server/runserver.go
@@ -72,6 +72,7 @@ const (
 	DefaultRefreshPrometheusMetricsInterval         = 5 * time.Second                  // default for --refreshPrometheusMetricsInterval
 	DefaultSecureServing                            = true                             // default for --secureServing
 	DefaultHealthChecking                           = false                            // default for --healthChecking
+	DefaultEnablePprof                              = true                             // default for --enablePprof
 	DefaultTotalQueuedRequestsMetric                = "vllm:num_requests_waiting"      // default for --totalQueuedRequestsMetric
 	DefaultKvCacheUsagePercentageMetric             = "vllm:gpu_cache_usage_perc"      // default for --kvCacheUsagePercentageMetric
 	DefaultLoraInfoMetric                           = "vllm:lora_requests_info"        // default for --loraInfoMetric



Test patch:
diff --git a/conformance/testing-epp/scheduler.go b/conformance/testing-epp/scheduler.go
deleted file mode 100644
index aaee956..0000000
--- a/conformance/testing-epp/scheduler.go
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
-Copyright 2025 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package scheduling
-
-import (
-	"sigs.k8s.io/gateway-api-inference-extension/conformance/testing-epp/plugins/filter"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
-)
-
-// NewReqHeaderBasedScheduler creates a scheduler for conformance tests that selects
-// an endpoint based on the "test-epp-endpoint-selection" request header. If the
-// header is missing or the specified endpoint doesn't exist, no endpoint is returned.
-func NewReqHeaderBasedScheduler() *scheduling.Scheduler {
-	predicatableSchedulerProfile := framework.NewSchedulerProfile().
-		WithFilters(filter.NewHeaderBasedTestingFilter()).
-		WithPicker(picker.NewMaxScorePicker(picker.DefaultMaxNumOfEndpoints))
-
-	return scheduling.NewSchedulerWithConfig(scheduling.NewSchedulerConfig(
-		profile.NewSingleProfileHandler(), map[string]*framework.SchedulerProfile{"req-header-based-profile": predicatableSchedulerProfile}))
-}
diff --git a/conformance/testing-epp/scheduler_test.go b/conformance/testing-epp/scheduler_test.go
deleted file mode 100644
index 95d627e..0000000
--- a/conformance/testing-epp/scheduler_test.go
+++ /dev/null
@@ -1,115 +0,0 @@
-/*
-Copyright 2025 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package scheduling
-
-import (
-	"context"
-	"testing"
-
-	"github.com/google/go-cmp/cmp"
-	"github.com/google/uuid"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
-	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics" // Import config for thresholds
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
-)
-
-// Tests the scheduler for conformance tests.
-func TestSchedule(t *testing.T) {
-	tests := []struct {
-		name    string
-		input   []types.Pod
-		req     *types.LLMRequest
-		wantRes *types.SchedulingResult
-		err     bool
-	}{
-		{
-			name: "no candidate pods and req header is set",
-			req: &types.LLMRequest{
-				Headers:   map[string]string{"test-epp-endpoint-selection": "random-endpoint"},
-				RequestId: uuid.NewString(),
-			},
-			wantRes: nil,
-			err:     true,
-		},
-		{
-			name: "req header not set",
-			input: []types.Pod{
-				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "random-endpoint"}},
-			},
-			req: &types.LLMRequest{
-				Headers:   map[string]string{}, // Deliberately set an empty header.
-				RequestId: uuid.NewString(),
-			},
-			wantRes: nil,
-			err:     true,
-		},
-		{
-			name: "no pods address from the candidate pods matches req header address",
-			input: []types.Pod{
-				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "nonmatched-endpoint"}},
-			},
-			req: &types.LLMRequest{
-				Headers:   map[string]string{"test-epp-endpoint-selection": "matched-endpoint"},
-				RequestId: uuid.NewString(),
-			},
-			wantRes: nil,
-			err:     true,
-		},
-		{
-			name: "one pod address from the candidate pods matches req header address",
-			input: []types.Pod{
-				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "nonmatched-endpoint"}},
-				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "matched-endpoint"}},
-			},
-			req: &types.LLMRequest{
-				Headers:   map[string]string{"test-epp-endpoint-selection": "matched-endpoint"},
-				RequestId: uuid.NewString(),
-			},
-			wantRes: &types.SchedulingResult{
-				ProfileResults: map[string]*types.ProfileRunResult{
-					"req-header-based-profile": {
-						TargetPods: []types.Pod{
-							&types.ScoredPod{
-								Pod: &types.PodMetrics{
-									Pod: &backend.Pod{
-										Address: "matched-endpoint",
-										Labels:  map[string]string{},
-									},
-								},
-							},
-						},
-					},
-				},
-				PrimaryProfileName: "req-header-based-profile",
-			},
-		},
-	}
-
-	for _, test := range tests {
-		t.Run(test.name, func(t *testing.T) {
-			scheduler := NewReqHeaderBasedScheduler()
-			got, err := scheduler.Schedule(context.Background(), test.req, test.input)
-			if test.err != (err != nil) {
-				t.Errorf("Unexpected error, got %v, want %v", err, test.err)
-			}
-
-			if diff := cmp.Diff(test.wantRes, got); diff != "" {
-				t.Errorf("Unexpected output (-want +got): %v", diff)
-			}
-		})
-	}
-}
diff --git a/pkg/epp/scheduling/framework/plugins/test/README.md b/pkg/epp/scheduling/framework/plugins/test/README.md
new file mode 100644
index 0000000..00c05db
--- /dev/null
+++ b/pkg/epp/scheduling/framework/plugins/test/README.md
@@ -0,0 +1 @@
+This package contains plugins implementation for test purpose only.
diff --git a/conformance/testing-epp/plugins/filter/filter_test.go b/pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go
similarity index 67%
rename from conformance/testing-epp/plugins/filter/filter_test.go
rename to pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go
index 2c00821..1eb1291 100644
--- a/conformance/testing-epp/plugins/filter/filter_test.go
+++ b/pkg/epp/scheduling/framework/plugins/test/filter/filter_test.go
@@ -22,7 +22,6 @@ import (
 
 	"github.com/google/go-cmp/cmp"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
@@ -30,14 +29,12 @@ func TestFilter(t *testing.T) {
 	tests := []struct {
 		name   string
 		req    *types.LLMRequest
-		filter framework.Filter
 		input  []types.Pod
 		output []types.Pod
 	}{
 		{
-			name:   "TestHeaderBasedFilter, header endpoint unset in request",
-			req:    &types.LLMRequest{}, // Delieverately unset the header.
-			filter: &HeaderBasedTestingFilter{},
+			name: "TestHeaderBasedFilter, header endpoint unset in request",
+			req:  &types.LLMRequest{}, // Delieverately unset the header.
 			input: []types.Pod{
 				&types.PodMetrics{
 					Pod: &backend.Pod{
@@ -48,9 +45,8 @@ func TestFilter(t *testing.T) {
 			output: []types.Pod{},
 		},
 		{
-			name:   "TestHeaderBasedFilter, header endpoint set in request but no match",
-			req:    &types.LLMRequest{Headers: map[string]string{headerTestEppEndPointSelectionKey: "test-endpoint"}},
-			filter: &HeaderBasedTestingFilter{},
+			name: "TestHeaderBasedFilter, header endpoint set in request but no match",
+			req:  &types.LLMRequest{Headers: map[string]string{headerTestEppEndPointSelectionKey: "test-endpoint"}},
 			input: []types.Pod{
 				&types.PodMetrics{
 					Pod: &backend.Pod{
@@ -61,9 +57,8 @@ func TestFilter(t *testing.T) {
 			output: []types.Pod{},
 		},
 		{
-			name:   "TestHeaderBasedFilter, header endpoint set",
-			req:    &types.LLMRequest{Headers: map[string]string{headerTestEppEndPointSelectionKey: "test-endpoint"}},
-			filter: &HeaderBasedTestingFilter{},
+			name: "TestHeaderBasedFilter, header endpoint set",
+			req:  &types.LLMRequest{Headers: map[string]string{headerTestEppEndPointSelectionKey: "test-endpoint"}},
 			input: []types.Pod{
 				&types.PodMetrics{
 					Pod: &backend.Pod{
@@ -80,9 +75,8 @@ func TestFilter(t *testing.T) {
 			},
 		},
 		{
-			name:   "TestHeaderBasedFilter, multiple header endpoints set and multiple matches",
-			req:    &types.LLMRequest{Headers: map[string]string{headerTestEppEndPointSelectionKey: "test-endpoint3,test-endpoint2"}},
-			filter: &HeaderBasedTestingFilter{},
+			name: "TestHeaderBasedFilter, multiple header endpoints set and multiple matches",
+			req:  &types.LLMRequest{Headers: map[string]string{headerTestEppEndPointSelectionKey: "test-endpoint3,test-endpoint2"}},
 			input: []types.Pod{
 				&types.PodMetrics{
 					Pod: &backend.Pod{
@@ -117,7 +111,7 @@ func TestFilter(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			got := test.filter.Filter(context.Background(), types.NewCycleState(), test.req, test.input)
+			got := NewHeaderBasedTestingFilter().Filter(context.Background(), types.NewCycleState(), test.req, test.input)
 
 			if diff := cmp.Diff(test.output, got); diff != "" {
 				t.Errorf("Unexpected output (-want +got): %v", diff)
diff --git a/conformance/testing-epp/plugins/filter/request_header_based_filter.go b/pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go
similarity index 79%
rename from conformance/testing-epp/plugins/filter/request_header_based_filter.go
rename to pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go
index 423ad26..4295b5f 100644
--- a/conformance/testing-epp/plugins/filter/request_header_based_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/test/filter/request_header_based_filter.go
@@ -18,6 +18,7 @@ package filter
 
 import (
 	"context"
+	"encoding/json"
 	"strings"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
@@ -31,16 +32,23 @@ const (
 	// E.g., "test-epp-endpoint-selection": "10.0.0.7,10.0.0.8"
 	// The returned order is the same as the order provided in the header.
 	headerTestEppEndPointSelectionKey = "test-epp-endpoint-selection"
+	// HeaderBasedTestingFilterType is the filter type that is used in plugins registry.
+	HeaderBasedTestingFilterType = "header-based-testing-filter"
 )
 
 // compile-time type assertion
 var _ framework.Filter = &HeaderBasedTestingFilter{}
 
+// HeaderBasedTestingFilterFactory defines the factory function for HeaderBasedTestingFilter.
+func HeaderBasedTestingFilterFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
+	return NewHeaderBasedTestingFilter().WithName(name), nil
+}
+
 // NewHeaderBasedTestingFilter initializes a new HeaderBasedTestingFilter.
 // This should only be used for testing purposes.
 func NewHeaderBasedTestingFilter() *HeaderBasedTestingFilter {
 	return &HeaderBasedTestingFilter{
-		typedName: plugins.TypedName{Type: "header-based-testing", Name: "header-based-testing-filter"},
+		typedName: plugins.TypedName{Type: HeaderBasedTestingFilterType, Name: HeaderBasedTestingFilterType},
 	}
 }
 
@@ -54,6 +62,12 @@ func (f *HeaderBasedTestingFilter) TypedName() plugins.TypedName {
 	return f.typedName
 }
 
+// WithName sets the name of the filter.
+func (f *HeaderBasedTestingFilter) WithName(name string) *HeaderBasedTestingFilter {
+	f.typedName.Name = name
+	return f
+}
+
 // Filter selects pods that match the IP addresses specified in the request header.
 func (f *HeaderBasedTestingFilter) Filter(_ context.Context, _ *types.CycleState, request *types.LLMRequest, pods []types.Pod) []types.Pod {
 	headerValue, ok := request.Headers[headerTestEppEndPointSelectionKey]