Repository: kubernetes-sigs/gateway-api-inference-extension

Issue:
extensible data layer: EPP should allow configurable metrics collection
once we add the notion of `Filter` and `Scorer` to EPP, I envisage that orgs that use GIE will have their own filters and scorers in some internal repo.
while GIE is not expected to support dynamic changes of filters/scorers, I believe we do want to allow an easy way to setup different filters/scorers in a fork. That is, I expect the filters/scorers to get registered in the code in a single place (e.g., single file), such that others can change their filters/scorers without drifting from upstream.
With that in mind, the main logic in a scorer (and/or filter) is usually based on some metric(s). so basically if one wants to define his own scorer, he might need to define also additional metric(s).
same as I expect that filters/scorers setup can be easily replaced without drifting from upstream, I think we should allow a very easy, simple and straight forward way to define the set of metrics (that eventually filter/scorer will use) that EPP collects.

Hints from issue comments:
@nirrozenbaum: seems related to this comment https://github.com/kubernetes-sigs/gateway-api-inference-extension/pull/683#discussion_r2048424522@ahg-g: Can we rename this to "pluggable/extensible data layer" please to align on the same terminology we are using for the scheduling pkg.@nirrozenbaum: I added a prefix "extensible data layer". 
please note that this does NOT cover the whole extensible data layer, but only the ability to be able to extend the metrics that are being collected.
data layer extensibility may include other extensions/collectors, like collecting v1/models from the endpoints.@elevran: Extensible data layer [proposal](https://docs.google.com/document/d/1eCCuyB_VW08ik_jqPC1__z6FzeWO_VOlPDUpN85g9Ww/edit?usp=sharing)@liu-cong: This issue is currently in the v0.4 release tracker and  v0.4 release is planned for Mid-June. 

@nirrozenbaum  @elevran It looks like this requires some discussion and isn't a blocker for v0.4. Should we remove this from v0.4 scope?@ahg-g: Yes, lets push this to 0.5 pls@elevran: I doubt we'll manage to review, revise and implement PR in the next week, so fully support pushing to 0.5@nirrozenbaum: /assign @elevran @nirrozenbaum: Hi @elevran, 
since first RC is planned in less than a week, I’m pinging issue owners for status update. 

where do we stand with this issue and what is the completion time estimation?@elevran: follow-up PR enabling data layer as _experimental_ feature should be up in 1-2 days.@danehans: @elevran any updates on the PR for this issue?@elevran: Hey @danehans - there have been a few in a series (e.g., #1351, #1237, #1195, #1154).
Expect one or two more PRs before this can be closed.
For the first phase, we'll keep the existing datalayer as default and enable the new implementation via feature flag.

Solution patch:
diff --git a/.github/ISSUE_TEMPLATE/new-release.md b/.github/ISSUE_TEMPLATE/new-release.md
index 27e8378..16ff164 100644
--- a/.github/ISSUE_TEMPLATE/new-release.md
+++ b/.github/ISSUE_TEMPLATE/new-release.md
@@ -35,10 +35,14 @@ This document defines the process for releasing Gateway API Inference Extension.
    export RC=1
    ```
 
-4. The vLLM image tag defaults to `v0.7.2` for a release. Set the `VLLM` environment variable if a newer [tag][vllm-tag] has been published. For example:
+4. Refer to the [release-quickstart script][release-quickstart] for the default image tags used
+   by the vLLM deployment manifests. If a newer [GPU][vllm-gpu-tag], [CPU][vllm-cpu-tag], or [Simulator][vllm-sim-tag]
+   tag has been published, set the appropriate environment variable or update the script. For example:
 
    ```shell
-   export VLLM=0.7.3
+   export VLLM_GPU=0.9.2
+   export VLLM_CPU=0.9.3
+   export VLLM_SIM=0.1.2
    ```
 
 ## Release Process
@@ -159,4 +163,7 @@ Use the following steps to announce the release.
 [k8s.io]: https://github.com/kubernetes/k8s.io
 [yaml]: https://github.com/kubernetes/k8s.io/blob/main/registry.k8s.io/images/k8s-staging-gateway-api-inference-extension/images.yaml
 [issue]: https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/new/choose
-[vllm-tag]: https://hub.docker.com/r/vllm/vllm-openai/tags
+[vllm-gpu-tag]: https://hub.docker.com/r/vllm/vllm-openai/tags
+[vllm-cpu-tag]: https://gallery.ecr.aws/q9t5s3a7/vllm-cpu-release-repo
+[vllm-sim-tag]: https://github.com/llm-d/llm-d-inference-sim/pkgs/container/llm-d-inference-sim
+[release-quickstart]: https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/hack/release-quickstart.sh
diff --git a/Makefile b/Makefile
index 9f5aacc..52fe9fd 100644
--- a/Makefile
+++ b/Makefile
@@ -337,7 +337,7 @@ release: artifacts release-quickstart verify test # Create a release.
 ## Location to install dependencies to
 LOCALBIN ?= $(shell pwd)/bin
 $(LOCALBIN):
-	mkdir -p $(LOCALBIN)
+	[ -d $@ ] || mkdir -p $@
 
 ## Tool Binaries
 KUBECTL ?= kubectl
diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES
index 933fbe9..32a0493 100644
--- a/OWNERS_ALIASES
+++ b/OWNERS_ALIASES
@@ -5,14 +5,13 @@ aliases:
   gateway-api-inference-extension-maintainers:
   - ahg-g
   - danehans
-  - jeffwan
+  - nirrozenbaum
   - kfswain
 
   gateway-api-inference-extension-reviewers:
   - liu-cong
   - robscott
   - shaneutt
-  - nirrozenbaum
 
 
   wg-serving-leads:
diff --git a/api/config/v1alpha1/defaults.go b/api/config/v1alpha1/defaults.go
index f59bc7c..07f1f9a 100644
--- a/api/config/v1alpha1/defaults.go
+++ b/api/config/v1alpha1/defaults.go
@@ -23,7 +23,7 @@ package v1alpha1
 func SetDefaults_EndpointPickerConfig(cfg *EndpointPickerConfig) {
 	for idx, pluginConfig := range cfg.Plugins {
 		if pluginConfig.Name == "" {
-			cfg.Plugins[idx].Name = pluginConfig.PluginName
+			cfg.Plugins[idx].Name = pluginConfig.Type
 		}
 	}
 }
diff --git a/api/config/v1alpha1/endpointpickerconfig_types.go b/api/config/v1alpha1/endpointpickerconfig_types.go
index 86f5ab7..92baa6c 100644
--- a/api/config/v1alpha1/endpointpickerconfig_types.go
+++ b/api/config/v1alpha1/endpointpickerconfig_types.go
@@ -46,13 +46,13 @@ type EndpointPickerConfig struct {
 type PluginSpec struct {
 	// +optional
 	// Name provides a name for plugin entries to reference. If
-	// omitted, the value of the PluginName field will be used.
+	// omitted, the value of the Plugin's Type field will be used.
 	Name string `json:"name"`
 
 	// +required
 	// +kubebuilder:validation:Required
-	// PluginName specifies the plugin to be instantiated.
-	PluginName string `json:"pluginName"`
+	// Type specifies the plugin type to be instantiated.
+	Type string `json:"type"`
 
 	// +optional
 	// Parameters are the set of parameters to be passed to the plugin's
diff --git a/api/v1alpha2/inferencepool_types.go b/api/v1alpha2/inferencepool_types.go
index 33759ba..fc7cfa3 100644
--- a/api/v1alpha2/inferencepool_types.go
+++ b/api/v1alpha2/inferencepool_types.go
@@ -31,7 +31,11 @@ type InferencePool struct {
 	metav1.TypeMeta   `json:",inline"`
 	metav1.ObjectMeta `json:"metadata,omitempty"`
 
-	Spec   InferencePoolSpec   `json:"spec,omitempty"`
+	Spec InferencePoolSpec `json:"spec,omitempty"`
+
+	// Status defines the observed state of InferencePool.
+	//
+	// +kubebuilder:default={parent: {{parentRef: {kind: "Status", name: "default"}, conditions: {{type: "Accepted", status: "Unknown", reason: "Pending", message: "Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}}}}
 	Status InferencePoolStatus `json:"status,omitempty"`
 }
 
@@ -150,14 +154,19 @@ const (
 	FailClose ExtensionFailureMode = "FailClose"
 )
 
-// InferencePoolStatus defines the observed state of InferencePool
+// InferencePoolStatus defines the observed state of InferencePool.
 type InferencePoolStatus struct {
 	// Parents is a list of parent resources (usually Gateways) that are
-	// associated with the route, and the status of the InferencePool with respect to
+	// associated with the InferencePool, and the status of the InferencePool with respect to
 	// each parent.
 	//
-	// A maximum of 32 Gateways will be represented in this list. An empty list
-	// means the route has not been attached to any Gateway.
+	// A maximum of 32 Gateways will be represented in this list. When the list contains
+	// `kind: Status, name: default`, it indicates that the InferencePool is not
+	// associated with any Gateway and a controller must perform the following:
+	//
+	//  - Remove the parent when setting the "Accepted" condition.
+	//  - Add the parent when the controller will no longer manage the InferencePool
+	//    and no other parents exist.
 	//
 	// +kubebuilder:validation:MaxItems=32
 	Parents []PoolStatus `json:"parent,omitempty"`
@@ -190,7 +199,7 @@ type InferencePoolConditionType string
 type InferencePoolReason string
 
 const (
-	// This condition indicates whether the route has been accepted or rejected
+	// This condition indicates whether the InferencePool has been accepted or rejected
 	// by a Gateway, and why.
 	//
 	// Possible reasons for this condition to be True are:
@@ -209,7 +218,7 @@ const (
 	// prefer to use the reasons listed above to improve interoperability.
 	InferencePoolConditionAccepted InferencePoolConditionType = "Accepted"
 
-	// This reason is used with the "Accepted" condition when the Route has been
+	// This reason is used with the "Accepted" condition when the InferencePool has been
 	// accepted by the Gateway.
 	InferencePoolReasonAccepted InferencePoolReason = "Accepted"
 
@@ -219,7 +228,7 @@ const (
 	InferencePoolReasonNotSupportedByGateway InferencePoolReason = "NotSupportedByGateway"
 
 	// This reason is used with the "Accepted" when a controller has not yet
-	// reconciled the route.
+	// reconciled the InferencePool.
 	InferencePoolReasonPending InferencePoolReason = "Pending"
 )
 
@@ -233,7 +242,7 @@ const (
 	//
 	// Possible reasons for this condition to be False are:
 	//
-	// * "InvalidExtnesionRef"
+	// * "InvalidExtensionRef"
 	//
 	// Controllers MAY raise this condition with other reasons, but should
 	// prefer to use the reasons listed above to improve interoperability.
diff --git a/cmd/epp/main.go b/cmd/epp/main.go
index 5d5b2c5..b5e0617 100644
--- a/cmd/epp/main.go
+++ b/cmd/epp/main.go
@@ -26,7 +26,9 @@ import (
 
 func main() {
 	// Register all known plugin factories
-	runner.RegisterAllPlgugins()
+	runner.RegisterAllPlugins()
+	// For adding out-of-tree plugins to the plugins registry, use the following:
+	// plugins.Register(my-out-of-tree-plugin-name, my-out-of-tree-plugin-factory-function)
 
 	if err := runner.NewRunner().Run(ctrl.SetupSignalHandler()); err != nil {
 		os.Exit(1)
diff --git a/cmd/epp/runner/register.go b/cmd/epp/runner/register.go
index ab525a2..e2ce601 100644
--- a/cmd/epp/runner/register.go
+++ b/cmd/epp/runner/register.go
@@ -25,20 +25,63 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/scorer"
 )
 
-// RegisterAllPlgugins registers the factory functions of all known plugins
-func RegisterAllPlgugins() {
-	plugins.Register(filter.LeastKVCacheFilterName, filter.LeastKVCacheFilterFactory)
-	plugins.Register(filter.LeastQueueFilterName, filter.LeastQueueFilterFactory)
-	plugins.Register(filter.LoraAffinityFilterName, filter.LoraAffinityFilterFactory)
-	plugins.Register(filter.LowQueueFilterName, filter.LowQueueFilterFactory)
-	plugins.Register(prefix.PrefixCachePluginName, prefix.PrefixCachePluginFactory)
-	plugins.Register(picker.MaxScorePickerName, picker.MaxScorePickerFactory)
-	plugins.Register(picker.RandomPickerName, picker.RandomPickerFactory)
-	plugins.Register(profile.SingleProfileHandlerName, profile.SingleProfileHandlerFactory)
-	plugins.Register(scorer.KvCacheScorerName, scorer.KvCacheScorerFactory)
-	plugins.Register(scorer.QueueScorerName, scorer.QueueScorerFactory)
-}
-
-// eppHandle is a temporary implementation of the interface plugins.Handle
+// RegisterAllPlugins registers the factory functions of all known plugins
+func RegisterAllPlugins() {
+	plugins.Register(filter.LeastKVCacheFilterType, filter.LeastKVCacheFilterFactory)
+	plugins.Register(filter.LeastQueueFilterType, filter.LeastQueueFilterFactory)
+	plugins.Register(filter.LoraAffinityFilterType, filter.LoraAffinityFilterFactory)
+	plugins.Register(filter.LowQueueFilterType, filter.LowQueueFilterFactory)
+	plugins.Register(prefix.PrefixCachePluginType, prefix.PrefixCachePluginFactory)
+	plugins.Register(picker.MaxScorePickerType, picker.MaxScorePickerFactory)
+	plugins.Register(picker.RandomPickerType, picker.RandomPickerFactory)
+	plugins.Register(profile.SingleProfileHandlerType, profile.SingleProfileHandlerFactory)
+	plugins.Register(scorer.KvCacheScorerType, scorer.KvCacheScorerFactory)
+	plugins.Register(scorer.QueueScorerType, scorer.QueueScorerFactory)
+}
+
+// eppHandle is an implementation of the interface plugins.Handle
 type eppHandle struct {
+	plugins plugins.HandlePlugins
+}
+
+// Plugins returns the sub-handle for working with instantiated plugins
+func (h *eppHandle) Plugins() plugins.HandlePlugins {
+	return h.plugins
+}
+
+// eppHandlePlugins implements the set of APIs to work with instantiated plugins
+type eppHandlePlugins struct {
+	thePlugins map[string]plugins.Plugin
+}
+
+// Plugin returns the named plugin instance
+func (h *eppHandlePlugins) Plugin(name string) plugins.Plugin {
+	return h.thePlugins[name]
+}
+
+// AddPlugin adds a plugin to the set of known plugin instances
+func (h *eppHandlePlugins) AddPlugin(name string, plugin plugins.Plugin) {
+	h.thePlugins[name] = plugin
+}
+
+// GetAllPlugins returns all of the known plugins
+func (h *eppHandlePlugins) GetAllPlugins() []plugins.Plugin {
+	result := make([]plugins.Plugin, 0)
+	for _, plugin := range h.thePlugins {
+		result = append(result, plugin)
+	}
+	return result
+}
+
+// GetAllPluginsWithNames returns al of the known plugins with their names
+func (h *eppHandlePlugins) GetAllPluginsWithNames() map[string]plugins.Plugin {
+	return h.thePlugins
+}
+
+func newEppHandle() *eppHandle {
+	return &eppHandle{
+		plugins: &eppHandlePlugins{
+			thePlugins: map[string]plugins.Plugin{},
+		},
+	}
 }
diff --git a/cmd/epp/runner/runner.go b/cmd/epp/runner/runner.go
index ea4fba0..7bb55b7 100644
--- a/cmd/epp/runner/runner.go
+++ b/cmd/epp/runner/runner.go
@@ -35,15 +35,13 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/metrics/filters"
 	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
 
-	"sigs.k8s.io/gateway-api-inference-extension/api/config/v1alpha1"
 	conformance_epp "sigs.k8s.io/gateway-api-inference-extension/conformance/testing-epp"
 	"sigs.k8s.io/gateway-api-inference-extension/internal/runnable"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config/loader"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datastore"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics/collectors"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/requestcontrol"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
@@ -217,32 +215,33 @@ func (r *Runner) Run(ctx context.Context) error {
 		return err
 	}
 
-	var theConfig *v1alpha1.EndpointPickerConfig
-	var instantiatedPlugins map[string]plugins.Plugin
-
 	if len(*configText) != 0 || len(*configFile) != 0 {
-		theConfig, err = config.LoadConfig([]byte(*configText), *configFile)
+		theConfig, err := loader.LoadConfig([]byte(*configText), *configFile)
 		if err != nil {
 			setupLog.Error(err, "Failed to load the configuration")
 			return err
 		}
 
-		epp := eppHandle{}
-		instantiatedPlugins, err = config.LoadPluginReferences(theConfig.Plugins, epp)
+		epp := newEppHandle()
+
+		err = loader.LoadPluginReferences(theConfig.Plugins, epp)
 		if err != nil {
 			setupLog.Error(err, "Failed to instantiate the plugins")
 			return err
 		}
 
-		r.schedulerConfig, err = scheduling.LoadSchedulerConfig(theConfig.SchedulingProfiles, instantiatedPlugins, setupLog)
+		r.schedulerConfig, err = loader.LoadSchedulerConfig(theConfig.SchedulingProfiles, epp)
 		if err != nil {
 			setupLog.Error(err, "Failed to create Scheduler configuration")
 			return err
 		}
+
+		// Add requestControl plugins
+		r.requestControlConfig.AddPlugins(epp.Plugins().GetAllPlugins()...)
 	}
 
 	// --- Initialize Core EPP Components ---
-	scheduler, err := r.initializeScheduler(datastore)
+	scheduler, err := r.initializeScheduler()
 	if err != nil {
 		setupLog.Error(err, "Failed to create scheduler")
 		return err
@@ -250,10 +249,6 @@ func (r *Runner) Run(ctx context.Context) error {
 
 	saturationDetector := saturationdetector.NewDetector(sdConfig, datastore, ctrl.Log)
 
-	// Add requestControl plugins
-	if instantiatedPlugins != nil {
-		r.requestControlConfig.AddPlugins(instantiatedPlugins)
-	}
 	director := requestcontrol.NewDirectorWithConfig(datastore, scheduler, saturationDetector, r.requestControlConfig)
 
 	// --- Setup ExtProc Server Runner ---
@@ -297,13 +292,13 @@ func (r *Runner) Run(ctx context.Context) error {
 	return nil
 }
 
-func (r *Runner) initializeScheduler(datastore datastore.Datastore) (*scheduling.Scheduler, error) {
+func (r *Runner) initializeScheduler() (*scheduling.Scheduler, error) {
 	if r.schedulerConfig != nil {
-		return scheduling.NewSchedulerWithConfig(datastore, r.schedulerConfig), nil
+		return scheduling.NewSchedulerWithConfig(r.schedulerConfig), nil
 	}
 
 	// otherwise, no one configured from outside scheduler config. use existing configuration
-	scheduler := scheduling.NewScheduler(datastore)
+	scheduler := scheduling.NewScheduler()
 	if schedulerV2 {
 		queueScorerWeight := envutil.GetEnvInt("QUEUE_SCORE_WEIGHT", scorer.DefaultQueueScorerWeight, setupLog)
 		kvCacheScorerWeight := envutil.GetEnvInt("KV_CACHE_SCORE_WEIGHT", scorer.DefaultKVCacheScorerWeight, setupLog)
@@ -321,11 +316,11 @@ func (r *Runner) initializeScheduler(datastore datastore.Datastore) (*scheduling
 		}
 
 		schedulerConfig := scheduling.NewSchedulerConfig(profile.NewSingleProfileHandler(), map[string]*framework.SchedulerProfile{"schedulerv2": schedulerProfile})
-		scheduler = scheduling.NewSchedulerWithConfig(datastore, schedulerConfig)
+		scheduler = scheduling.NewSchedulerWithConfig(schedulerConfig)
 	}
 
 	if reqHeaderBasedSchedulerForTesting {
-		scheduler = conformance_epp.NewReqHeaderBasedScheduler(datastore)
+		scheduler = conformance_epp.NewReqHeaderBasedScheduler()
 	}
 
 	return scheduler, nil
diff --git a/config/charts/inferencepool/templates/rbac.yaml b/config/charts/inferencepool/templates/rbac.yaml
index cdd50c6..4cb5d7b 100644
--- a/config/charts/inferencepool/templates/rbac.yaml
+++ b/config/charts/inferencepool/templates/rbac.yaml
@@ -33,6 +33,7 @@ subjects:
   name: {{ include "gateway-api-inference-extension.name" . }}
   namespace: {{ .Release.Namespace }}
 roleRef:
+  apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: {{ include "gateway-api-inference-extension.name" . }}
 ---
diff --git a/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml b/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
index 8386db8..db342d0 100644
--- a/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
+++ b/config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
@@ -134,16 +134,32 @@ spec:
             - targetPortNumber
             type: object
           status:
-            description: InferencePoolStatus defines the observed state of InferencePool
+            default:
+              parent:
+              - conditions:
+                - lastTransitionTime: "1970-01-01T00:00:00Z"
+                  message: Waiting for controller
+                  reason: Pending
+                  status: Unknown
+                  type: Accepted
+                parentRef:
+                  kind: Status
+                  name: default
+            description: Status defines the observed state of InferencePool.
             properties:
               parent:
                 description: |-
                   Parents is a list of parent resources (usually Gateways) that are
-                  associated with the route, and the status of the InferencePool with respect to
+                  associated with the InferencePool, and the status of the InferencePool with respect to
                   each parent.
 
-                  A maximum of 32 Gateways will be represented in this list. An empty list
-                  means the route has not been attached to any Gateway.
+                  A maximum of 32 Gateways will be represented in this list. When the list contains
+                  `kind: Status, name: default`, it indicates that the InferencePool is not
+                  associated with any Gateway and a controller must perform the following:
+
+                   - Remove the parent when setting the "Accepted" condition.
+                   - Add the parent when the controller will no longer manage the InferencePool
+                     and no other parents exist.
                 items:
                   description: PoolStatus defines the observed state of InferencePool
                     from a Gateway.
diff --git a/config/manifests/inferencepool-resources.yaml b/config/manifests/inferencepool-resources.yaml
index 37099c4..34aacfa 100644
--- a/config/manifests/inferencepool-resources.yaml
+++ b/config/manifests/inferencepool-resources.yaml
@@ -117,5 +117,6 @@ subjects:
   name: default
   namespace: default
 roleRef:
+  apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: pod-read
diff --git a/config/manifests/vllm/sim-deployment.yaml b/config/manifests/vllm/sim-deployment.yaml
index d9706ea..1862efe 100644
--- a/config/manifests/vllm/sim-deployment.yaml
+++ b/config/manifests/vllm/sim-deployment.yaml
@@ -14,7 +14,7 @@ spec:
     spec:
       containers:
       - name: vllm-sim
-        image: ghcr.io/llm-d/llm-d-inference-sim:v0.1.1
+        image: ghcr.io/llm-d/llm-d-inference-sim:latest
         imagePullPolicy: Always
         args:
         - --model
diff --git a/conformance/conformance.go b/conformance/conformance.go
index d120062..406124b 100644
--- a/conformance/conformance.go
+++ b/conformance/conformance.go
@@ -45,7 +45,6 @@ import (
 	// Import necessary types and utilities from the core Gateway API conformance suite.
 	gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"            // Import core Gateway API types
 	confapis "sigs.k8s.io/gateway-api/conformance/apis/v1" // Report struct definition
-	confconfig "sigs.k8s.io/gateway-api/conformance/utils/config"
 	confflags "sigs.k8s.io/gateway-api/conformance/utils/flags"
 	apikubernetes "sigs.k8s.io/gateway-api/conformance/utils/kubernetes"
 	confsuite "sigs.k8s.io/gateway-api/conformance/utils/suite"
@@ -167,7 +166,7 @@ func DefaultOptions(t *testing.T) confsuite.ConformanceOptions {
 		Debug:                *confflags.ShowDebug,
 		CleanupBaseResources: *confflags.CleanupBaseResources,
 		SupportedFeatures:    sets.New[features.FeatureName](),
-		TimeoutConfig:        confconfig.DefaultTimeoutConfig(),
+		TimeoutConfig:        inferenceconfig.DefaultInferenceExtensionTimeoutConfig().TimeoutConfig,
 		SkipTests:            skipTests,
 		ExemptFeatures:       exemptFeatures,
 		RunTest:              *confflags.RunTest,
@@ -177,6 +176,7 @@ func DefaultOptions(t *testing.T) confsuite.ConformanceOptions {
 		ManifestFS:           []fs.FS{&Manifests},
 		ReportOutputPath:     *confflags.ReportOutput,
 		SkipProvisionalTests: *confflags.SkipProvisionalTests,
+		AllowCRDsMismatch:    *confflags.AllowCRDsMismatch,
 		NamespaceLabels:      namespaceLabels,
 		NamespaceAnnotations: namespaceAnnotations,
 		// TODO: Add the inference extension specific fields to ConformanceOptions struct if needed,
diff --git a/conformance/resources/manifests/manifests.yaml b/conformance/resources/manifests/manifests.yaml
index c4f9b1f..5b1f2d4 100644
--- a/conformance/resources/manifests/manifests.yaml
+++ b/conformance/resources/manifests/manifests.yaml
@@ -66,6 +66,7 @@ metadata:
   labels:
     app: primary-inference-model-server
 spec:
+  replicas: 3
   selector:
     matchLabels:
       app: primary-inference-model-server
@@ -109,6 +110,7 @@ metadata:
   labels:
     app: secondary-inference-model-server
 spec:
+  replicas: 3
   selector:
     matchLabels:
       app: secondary-inference-model-server
@@ -244,6 +246,7 @@ spec:
   targetPortNumber: 3000
   extensionRef:
     name: secondary-endpoint-picker-svc
+    failureMode: FailOpen
 ---
 # --- Secondary Conformance EPP service Definition ---
 apiVersion: v1
diff --git a/conformance/utils/config/timing.go b/conformance/utils/config/timing.go
index f520ec7..861eb0f 100644
--- a/conformance/utils/config/timing.go
+++ b/conformance/utils/config/timing.go
@@ -20,6 +20,7 @@ import (
 	"time"
 
 	// Import the upstream Gateway API timeout config
+
 	gatewayconfig "sigs.k8s.io/gateway-api/conformance/utils/config"
 )
 
@@ -44,8 +45,12 @@ type InferenceExtensionTimeoutConfig struct {
 
 // DefaultInferenceExtensionTimeoutConfig returns a new InferenceExtensionTimeoutConfig with default values.
 func DefaultInferenceExtensionTimeoutConfig() InferenceExtensionTimeoutConfig {
+	config := gatewayconfig.DefaultTimeoutConfig()
+	config.HTTPRouteMustHaveCondition = 300 * time.Second
+	config.MaxTimeToConsistency = 200 * time.Second
+	config.DefaultTestTimeout = 600 * time.Second
 	return InferenceExtensionTimeoutConfig{
-		TimeoutConfig:                          gatewayconfig.DefaultTimeoutConfig(),
+		TimeoutConfig:                          config, // Initialize embedded struct
 		GeneralMustHaveConditionTimeout:        300 * time.Second,
 		InferencePoolMustHaveConditionInterval: 10 * time.Second,
 		GatewayObjectPollInterval:              5 * time.Second,
diff --git a/conformance/utils/kubernetes/helpers.go b/conformance/utils/kubernetes/helpers.go
index f5cef75..2fcaa0c 100644
--- a/conformance/utils/kubernetes/helpers.go
+++ b/conformance/utils/kubernetes/helpers.go
@@ -26,6 +26,7 @@ import (
 	"time"
 
 	"github.com/stretchr/testify/require"
+	appsv1 "k8s.io/api/apps/v1"
 	corev1 "k8s.io/api/core/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -308,6 +309,28 @@ func GetGatewayEndpoint(t *testing.T, k8sClient client.Client, timeoutConfig gat
 	return gwAddr
 }
 
+// GetPodsWithLabel retrieves a list of Pods.
+// It finds pods matching the given labels in a specific namespace.
+func GetPodsWithLabel(t *testing.T, c client.Client, namespace string, labels map[string]string) ([]corev1.Pod, error) {
+	t.Helper()
+
+	podList := &corev1.PodList{}
+	listOptions := []client.ListOption{
+		client.InNamespace(namespace),
+		client.MatchingLabels(labels),
+	}
+
+	t.Logf("Searching for Pods with labels %v in namespace %s", labels, namespace)
+	if err := c.List(context.Background(), podList, listOptions...); err != nil {
+		return nil, fmt.Errorf("failed to list pods with labels '%v' in namespace '%s': %w", labels, namespace, err)
+	}
+
+	if len(podList.Items) == 0 {
+		return nil, fmt.Errorf("no pods found with labels '%v' in namespace '%s'", labels, namespace)
+	}
+	return podList.Items, nil
+}
+
 // GetPod waits for a Pod matching the specified labels to exist in the given
 // namespace and have an IP address assigned. This function returns the first
 // matching Pod found if there are multiple matches. It fails the on timeout or error.
@@ -339,7 +362,6 @@ func GetPod(t *testing.T, c client.Client, namespace string, selector labels.Sel
 		}
 		return false, nil
 	})
-
 	require.NoErrorf(t, waitErr, "timed out waiting for Pod with selector %s in namespace %s to be ready", selector.String(), namespace)
 	require.NotEmpty(t, pods.Items, "expected at least one pod for selector %s in namespace %s, but found none", selector.String(), namespace)
 
@@ -347,3 +369,43 @@ func GetPod(t *testing.T, c client.Client, namespace string, selector labels.Sel
 	t.Logf("Successfully found ready Pod %s with IP %s for selector %s", pod.Name, pod.Status.PodIP, selector.String())
 	return pod
 }
+
+// DeleteDeployment deletes the specified Deployment and waits until it is no longer
+// present in the cluster.
+func DeleteDeployment(t *testing.T, c client.Client, timeoutConfig gatewayapiconfig.TimeoutConfig, deploymentRef types.NamespacedName) error {
+	t.Helper()
+
+	deploymentToDelete := &appsv1.Deployment{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      deploymentRef.Name,
+			Namespace: deploymentRef.Namespace,
+		},
+	}
+
+	t.Logf("Deleting Deployment %s/%s...", deploymentRef.Namespace, deploymentRef.Name)
+	if err := c.Delete(context.Background(), deploymentToDelete); err != nil {
+		// If the resource is already gone, we don't consider it an error.
+		if !apierrors.IsNotFound(err) {
+			return fmt.Errorf("failed to delete Deployment %s/%s: %w", deploymentRef.Namespace, deploymentRef.Name, err)
+		}
+	}
+
+	// Wait for the Deployment to be fully removed.
+	waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.DeleteTimeout, true, func(ctx context.Context) (bool, error) {
+		var dep appsv1.Deployment
+		err := c.Get(ctx, deploymentRef, &dep)
+		if apierrors.IsNotFound(err) {
+			return true, nil
+		}
+		if err != nil {
+			return false, fmt.Errorf("error waiting for Deployment %s/%s to be deleted: %w", deploymentRef.Namespace, deploymentRef.Name, err)
+		}
+		return false, nil
+	})
+
+	if waitErr != nil {
+		return fmt.Errorf("timed out waiting for Deployment %s/%s to be deleted: %w", deploymentRef.Namespace, deploymentRef.Name, waitErr)
+	}
+	t.Logf("Successfully deleted Deployment %s/%s", deploymentRef.Namespace, deploymentRef.Name)
+	return nil
+}
diff --git a/conformance/utils/traffic/traffic.go b/conformance/utils/traffic/traffic.go
index 07f9449..e277223 100644
--- a/conformance/utils/traffic/traffic.go
+++ b/conformance/utils/traffic/traffic.go
@@ -17,16 +17,47 @@ limitations under the License.
 package traffic
 
 import (
+	"context"
+	"encoding/json"
 	"fmt"
+	"io"
 	"net/http"
+	"net/http/httputil"
+	"regexp"
+	"strings"
 	"testing"
+	"time"
 
 	corev1 "k8s.io/api/core/v1"
 	gwconfig "sigs.k8s.io/gateway-api/conformance/utils/config"
 	gwhttp "sigs.k8s.io/gateway-api/conformance/utils/http"
 	"sigs.k8s.io/gateway-api/conformance/utils/roundtripper"
+	"sigs.k8s.io/gateway-api/conformance/utils/tlog"
 )
 
+// Request defines the parameters for a single HTTP test request and its expected outcome.
+type Request struct {
+	// Host is the hostname to use in the HTTP request.
+	Host string
+	// Path is the path to request.
+	Path string
+	// Method is the HTTP method to use. Defaults to "GET" if empty.
+	Method string
+	// Headers are the HTTP headers to include in the request.
+	Headers map[string]string
+	// Body is the request body.
+	Body string
+
+	// ExpectedStatusCode is the HTTP status code expected in the response.
+	ExpectedStatusCode int
+	// Backend is the name of the backend service expected to handle the request.
+	// This is not checked for non-200 responses.
+	Backend string
+	// Namespace is the namespace of the backend service.
+	Namespace string
+}
+
+// Deprecated: this will be replaced by letting caller to construct the Request type.
 // BuildExpectedHTTPResponse constructs a gwhttp.ExpectedResponse for common test scenarios.
 // For 200 OK responses, it sets up ExpectedRequest to check Host and Path.
 // For other status codes (like 404), ExpectedRequest is nil as detailed backend checks are usually skipped by CompareRequest.
@@ -62,6 +93,8 @@ func BuildExpectedHTTPResponse(
 	return resp
 }
 
+// Deprecated: please use MakeRequestWithRequestParamAndExpectSuccess instead.
+// https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/983
 // MakeRequestAndExpectSuccess is a helper function that builds an expected success (200 OK) response
 // and then calls MakeRequestAndExpectEventuallyConsistentResponse.
 func MakeRequestAndExpectSuccess(
@@ -85,6 +118,8 @@ func MakeRequestAndExpectSuccess(
 	gwhttp.MakeRequestAndExpectEventuallyConsistentResponse(t, r, timeoutConfig, gatewayAddress, expectedResponse)
 }
 
+// Deprecated: please use MakeRequestAndExpectEventuallyConsistentResponse instead and specify the ExpectedStatusCode in Request.
+// https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/983
 // MakeRequestAndExpectNotFound is a helper function that builds an expected not found (404) response
 // and then calls MakeRequestAndExpectEventuallyConsistentResponse.
 func MakeRequestAndExpectNotFound(
@@ -106,6 +141,35 @@ func MakeRequestAndExpectNotFound(
 	gwhttp.MakeRequestAndExpectEventuallyConsistentResponse(t, r, timeoutConfig, gatewayAddress, expectedResponse)
 }
 
+// MakeRequestAndExpectEventuallyConsistentResponse makes a request using the parameters
+// from the Request struct and waits for the response to consistently match the expectations.
+func MakeRequestAndExpectEventuallyConsistentResponse(
+	t *testing.T,
+	r roundtripper.RoundTripper,
+	timeoutConfig gwconfig.TimeoutConfig,
+	gatewayAddress string,
+	req Request,
+) {
+	t.Helper()
+
+	expectedResponse := makeExpectedResponse(t, req)
+	waitForConvergeToExpected(t, r, timeoutConfig, gatewayAddress, req.Body, expectedResponse)
+}
+
+// MakeRequestWithRequestParamAndExpectSuccess is a convenience wrapper for requests that are
+// expected to succeed with a 200 OK status.
+func MakeRequestWithRequestParamAndExpectSuccess(
+	t *testing.T,
+	r roundtripper.RoundTripper,
+	timeoutConfig gwconfig.TimeoutConfig,
+	gatewayAddress string,
+	req Request,
+) {
+	t.Helper()
+	req.ExpectedStatusCode = http.StatusOK
+	MakeRequestAndExpectEventuallyConsistentResponse(t, r, timeoutConfig, gatewayAddress, req)
+}
+
 // MakeRequestAndExpectResponseFromPod sends a request to the specified path by IP address and
 // uses a special "test-epp-endpoint-selection" header to target a specific backend Pod.
 // It then verifies that the response was served by that Pod.
@@ -130,3 +194,208 @@ func MakeRequestAndExpectResponseFromPod(t *testing.T, r roundtripper.RoundTripp
 
 	gwhttp.MakeRequestAndExpectEventuallyConsistentResponse(t, r, timeoutConfig, gwAddr, expectedResponse)
 }
+
+func makeExpectedResponse(t *testing.T, req Request) gwhttp.ExpectedResponse {
+	t.Helper()
+
+	method := http.MethodGet
+	if req.Method != "" {
+		method = req.Method
+	}
+
+	expectedResponse := gwhttp.ExpectedResponse{
+		Request: gwhttp.Request{
+			Host:    req.Host,
+			Path:    req.Path,
+			Method:  method,
+			Headers: req.Headers,
+		},
+		Response: gwhttp.Response{
+			StatusCode: req.ExpectedStatusCode,
+		},
+		Backend:   req.Backend,
+		Namespace: req.Namespace,
+	}
+
+	// For successful responses (200 OK), we also verify that the backend
+	// received the request with the correct details (Host, Path, etc.).
+	// For other statuses (e.g., 404), this check is skipped.
+	if req.ExpectedStatusCode == http.StatusOK {
+		expectedResponse.ExpectedRequest = &gwhttp.ExpectedRequest{
+			Request: gwhttp.Request{
+				Host:    req.Host,
+				Path:    req.Path,
+				Headers: req.Headers,
+				Method:  method,
+			},
+		}
+	}
+	return expectedResponse
+}
+
+// TODO: https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/1031
+// replace the following method when sigs.k8s.io/gateway-api/conformance/utils/roundtripper is able to send request with body.
+func waitForConvergeToExpected(
+	t *testing.T,
+	r roundtripper.RoundTripper,
+	timeoutConfig gwconfig.TimeoutConfig,
+	gatewayAddress string,
+	requestBody string,
+	expectedResponse gwhttp.ExpectedResponse,
+) {
+	gwhttp.AwaitConvergence(t, timeoutConfig.RequiredConsecutiveSuccesses, timeoutConfig.MaxTimeToConsistency, func(elapsed time.Duration) bool {
+		req := gwhttp.MakeRequest(t, &expectedResponse, gatewayAddress, "HTTP", "http")
+		request := &RequestWithBody{Request: req}
+		if requestBody != "" {
+			request = &RequestWithBody{Request: req, Body: strings.NewReader(requestBody)}
+		}
+		cReq, cRes, err := MakeCallRoundTripper(t, r, request)
+		if err != nil {
+			tlog.Logf(t, "Request failed, not ready yet: %v (after %v)", err.Error(), elapsed)
+			return false
+		}
+
+		if err := gwhttp.CompareRequest(t, &request.Request, cReq, cRes, expectedResponse); err != nil {
+			tlog.Logf(t, "Response expectation failed for request: %+v  not ready yet: %v (after %v)", request.Request, err, elapsed)
+			return false
+		}
+
+		return true
+	})
+	tlog.Logf(t, "Request passed")
+}
+
+// TODO: https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/1031
+// remove this when sigs.k8s.io/gateway-api/conformance/utils/roundtripper is able to send request with body.
+// RequestWithBody extends roundtripper.Request to include a request body.
+type RequestWithBody struct {
+	roundtripper.Request
+	Body io.Reader
+}
+
+// TODO: https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/1031
+// remove this when sigs.k8s.io/gateway-api/conformance/utils/roundtripper is able to send request with body.
+// MakeCallRoundTripper executes an HTTP request using the provided RoundTripper and captures the request and response.
+func MakeCallRoundTripper(t *testing.T, r roundtripper.RoundTripper, request *RequestWithBody) (*roundtripper.CapturedRequest, *roundtripper.CapturedResponse, error) {
+	client := &http.Client{}
+
+	defaultRoundTripper, ok := r.(*roundtripper.DefaultRoundTripper)
+	if !ok {
+		t.Fatalf("Unsupported RoundTripper type: %T", r)
+	}
+	rt := defaultRoundTripper
+	if request.UnfollowRedirect {
+		client.CheckRedirect = func(_ *http.Request, _ []*http.Request) error {
+			return http.ErrUseLastResponse
+		}
+	}
+
+	client.Transport = &http.Transport{
+		DialContext: rt.CustomDialContext,
+		// We disable keep-alives so that we don't leak established TCP connections.
+		// Leaking TCP connections is bad because we could eventually hit the
+		// threshold of maximum number of open TCP connections to a specific
+		// destination. Keep-alives are not presently utilized so disabling this has
+		// no adverse affect.
+		//
+		// Ref. https://github.com/kubernetes-sigs/gateway-api/issues/2357
+		DisableKeepAlives: true,
+	}
+
+	method := "GET"
+	if request.Method != "" {
+		method = request.Method
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), rt.TimeoutConfig.RequestTimeout)
+	defer cancel()
+	req, err := http.NewRequestWithContext(ctx, method, request.URL.String(), request.Body)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	if request.Host != "" {
+		req.Host = request.Host
+	}
+
+	if request.Headers != nil {
+		for name, value := range request.Headers {
+			req.Header.Set(name, value[0])
+		}
+	}
+
+	if rt.Debug {
+		var dump []byte
+		dump, err = httputil.DumpRequestOut(req, true)
+		if err != nil {
+			return nil, nil, err
+		}
+
+		tlog.Logf(request.T, "Sending Request:\n%s\n\n", formatDump(dump, "< "))
+	}
+
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	if rt.Debug {
+		var dump []byte
+		dump, err = httputil.DumpResponse(resp, true)
+		if err != nil {
+			return nil, nil, err
+		}
+
+		tlog.Logf(request.T, "Received Response:\n%s\n\n", formatDump(dump, "< "))
+	}
+
+	cReq := &roundtripper.CapturedRequest{}
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	// we cannot assume the response is JSON
+	if resp.Header.Get("Content-type") == "application/json" {
+		err = json.Unmarshal(body, cReq)
+		if err != nil {
+			return nil, nil, fmt.Errorf("unexpected error reading response: %w", err)
+		}
+	} else {
+		cReq.Method = method // assume it made the right request if the service being called isn't echoing
+	}
+
+	cRes := &roundtripper.CapturedResponse{
+		StatusCode:    resp.StatusCode,
+		ContentLength: resp.ContentLength,
+		Protocol:      resp.Proto,
+		Headers:       resp.Header,
+	}
+
+	if resp.TLS != nil {
+		cRes.PeerCertificates = resp.TLS.PeerCertificates
+	}
+
+	if roundtripper.IsRedirect(resp.StatusCode) {
+		redirectURL, err := resp.Location()
+		if err != nil {
+			return nil, nil, err
+		}
+		cRes.RedirectRequest = &roundtripper.RedirectRequest{
+			Scheme: redirectURL.Scheme,
+			Host:   redirectURL.Hostname(),
+			Port:   redirectURL.Port(),
+			Path:   redirectURL.Path,
+		}
+	}
+
+	return cReq, cRes, nil
+}
+
+var startLineRegex = regexp.MustCompile(`(?m)^`)
+
+func formatDump(data []byte, prefix string) string {
+	data = startLineRegex.ReplaceAllLiteral(data, []byte(prefix))
+	return string(data)
+}
diff --git a/docs/proposals/0845-scheduler-architecture-proposal/interfaces/interface.go b/docs/proposals/0845-scheduler-architecture-proposal/interfaces/interface.go
index 6fd19e6..35b787b 100644
--- a/docs/proposals/0845-scheduler-architecture-proposal/interfaces/interface.go
+++ b/docs/proposals/0845-scheduler-architecture-proposal/interfaces/interface.go
@@ -85,7 +85,7 @@ type SchedulingResult struct {
 
 // Plugin is the parent type for all the scheduling framework plugins.
 type Plugin interface {
-	Name() string
+	Type() string
 }
 
 // ProfileHandler defines the interface for handling multi SchedulerProfile instances.
diff --git a/docs/proposals/1023-data-layer-architecture/README.md b/docs/proposals/1023-data-layer-architecture/README.md
new file mode 100644
index 0000000..7389fdb
--- /dev/null
+++ b/docs/proposals/1023-data-layer-architecture/README.md
@@ -0,0 +1,174 @@
+# Data Layer Architecture Proposal
+
+Author(s): @elevran @nirrozenbaum
+
+## Proposal Status
+
+***Draft***
+
+## Summary
+
+The EPP Architecture proposal identifies the need for an extensible
+ [Data Layer](../0683-epp-architecture-proposal/README.md#data-layer).
+ Recently, the scheduling subsystem underwent a major [architecure change](../0845-scheduler-architecture-proposal/README.md)
+ to allow easier extension and pluggability. This proposal aims to apply
+ similar extensibility to the Data Layer subsystem, allowing custom inference
+ gateways to extend the Gateway API Inference extension (GIE) for their use
+ cases without modifying the core GIE code base.
+
+See [this document](https://docs.google.com/document/d/1eCCuyB_VW08ik_jqPC1__z6FzeWO_VOlPDUpN85g9Ww/edit?usp=sharing) for additional context amd reference.
+
+## Goals
+
+The Data Layer pluggability effort aims to address the folllowing goals and
+ requirements:
+
+- Make endpoint attributes used by GIE components accessible via well defined
+  Data Layer interfaces.
+- Enable collection of additional (or different) subset of attributes from an
+  existing data source (e.g., the `/metrics` endpoint scraper).
+- Add a new data source that collects attributes not already collected.
+- Follow best practices and experience from the Scheduling subsystem
+  pluggability effort. For example, extending the system to support the above
+  should be through implementing well defined Plugin interfaces and registering
+  them in the GIE Data Layer subsystem; any configuration would be done in the
+  same way (e.g., code and/or configuration file), etc.
+- Be efficient (RAM, CPU, concurrency) in collecting and storing attributes.
+- Limit change blast radius in GIE when making above changes. Core GIE code
+  should not need to be modified in order to support collecting and storing new
+  attributes. Affected code should be scoped only to modules that make use of
+  the new attributes.
+- The extensions should not increase coupling between GIE subsystems and
+  Kubernetes (i.e., the environment specific code should be encapsulated and
+  not “leaked” into the subsystem and its users).
+- (Future) Allow non-uniform data collection (i.e., not all endpoints share the
+  same data).
+
+## Non-Goals
+
+- Modify existing GIE abstractions, such as `InferencePool`, to conform to the
+  Data Layer pluggability design. They are to remain first class concepts, as
+  today.
+- Enable reconciliation or modification of external state. The data sources are
+  strictly read-only. For example, data source accessing Kubernetes state as part of
+  the data collection would registered for `Watch()` notifications and shall not
+  receive access to a k8s client.
+- Inference scheduler Plugins, that rely on custom data collection, accept that
+  the [Model Server Protocol](../003-model-server-protocol/README.md) no longer
+  provides guarantees on portability of a model server out of the box.
+
+## Proposal
+
+### Overview
+
+There are two existing Data Sources in the Data Layer: a Pod reconciler that
+ collects Pod IP address(es) and labels, copying them to endpoint attributes,
+ and a metrics scraper that collects a defined set of metric values from the
+ `/metrics` endpoint of each Pod. Note that the `InferencePool` reconciler is
+ *not* considered part of the Data Layer.
+
+### Components
+
+The proposal is to make the Data Layer more extensible approaching by introducing
+ these two interfaces:
+
+- An **Attribute Collection** plugin interface responsible for extratcing relevant
+  attributes from a data source and storing them into the Data Layer for consumption
+  by other components. The plugin can be registered with existing or new
+  *Data Sources* (see below) and sources would call their registered plugins
+  periodically or on change to process attributes.
+- A **Data source** plugin interface that can be added to an inference gateway
+  system, and on which *Attribute Collection* plugins can be registered to enrich
+  the data model.
+
+### Implementation Phases
+
+In order to make iterative progress and validate the design alongside, we
+ propose to implement and evolve the Data Layer extensibility over several
+ phases:
+
+1. Extend the backend, per endpoint, storage with a map from a name (i.e., the
+  attribute collection interface) to the data it collected. Existing attributes,
+  such as IP address or Pod labels, are not modified.
+1. Introduce a Data Source registry where new data sources can be registered and
+  bootstrap it by wrapping the existing `/metrics` with a Data Source API. At this
+  point, the metrics scraping code includes only the `Data Source` interface and the
+  `Data Collection` interface is not used/exposed.
+1. Refactor the metrics scraping code into separate Data Source and Data Collection
+  plugin interfaces.
+1. Following that, and based on any lessons learnt, we’ll refactor the existing
+  Kubernetes Pod recocilation loop to the new plugin interfaces.
+
+### Suggested Data Layer Plugin Interfaces
+
+```go
+// DataCollection interface consumes data updates from sources, stores
+// it in the data layer for consumption.
+// The plugin should not assume a deterministic invocation behavior beyond
+// "the data layer believes the state should be updated"
+type DataCollection interface {
+    // Extract is called by data sources with (possibly) updated
+    // data per endpoint. Extracted attributes are added to the 
+    // Endpoint.
+    Extract(ep Endpoint, data interface{}) error // or Collect?
+}
+
+// Endpoint interface allows setting and retrieving of attributes
+// by a data collector.
+// Note that actual endpoint structure would be something like (pseudocode)
+// type EndpointState struct {
+//   address
+//   ...
+//   data map[string]interface{}
+// }
+// The plugin interface would only mutate the `data` map
+type Endpoint interface {
+   // StoreAttributes sets the data for the Endpoint on behalf
+   // of the named collection Plugin
+   StoreAttributes(collector string, data interface{}) error
+   
+   // GetAttributes retrieves the attributes of the named collection
+   // plugin for the Endpoint
+   GetAttributes(collector string) (interface{}, error)
+}
+
+// DataLayerSourcesRegistry include the list of available 
+// Data Sources (interface defined below) in the system.
+// It is accompanied by functions (not shown) to register
+// and retrieve sources
+type DataLayerSourcesRegistry map[string]DataSource 
+
+// DataSource interface represents a data source that tracks
+// pods/resources and notifies data collection plugins to
+// extract relevant attributes.
+type DataSource interface {
+    // Type of data available from this source
+    Type() string
+
+    // Start begins the data collection and notification loop
+    Start(ctx context) error
+
+    // Stop terminates data collection
+    Stop() error
+
+    // Subscribe a collector to receive updates for tracked endpoints
+    Subscribe(collector DataCollection) error
+
+    // UpdateEndpoints replaces the set of pods/resources tracked by
+    // this source.
+    // Alternative: add/remove individual endpoints?
+    UpdateEndpoints(epIDs []string) error 
+}
+```
+
+## Open Questions
+
+1. Type safety in extensible data colletion: `map[string]interface{}` seems
+  like the simplest option to start, but may want to evolve to support
+  type safety using generics or codegen.
+1. Should we design a separate interface specifically for k8s object watching
+  under GIE control or do we want these to be managed as yet another data source?
+  This affects the design (e.g., who owns the k8s caches, clients, etc.).
+  With a GIE controlled data source, collectors just register the types (and
+  other constraints? Labels, namespaces, …) with GIE core, and all k8s
+  functionality is under GIE control.
diff --git a/go.mod b/go.mod
index 0c02dac..5e04947 100644
--- a/go.mod
+++ b/go.mod
@@ -14,18 +14,19 @@ require (
 	github.com/onsi/gomega v1.37.0
 	github.com/prometheus/client_golang v1.22.0
 	github.com/prometheus/client_model v0.6.2
-	github.com/prometheus/common v0.64.0
+	github.com/prometheus/common v0.65.0
 	github.com/stretchr/testify v1.10.0
 	go.uber.org/multierr v1.11.0
 	go.uber.org/zap v1.27.0
+	golang.org/x/sync v0.14.0
 	google.golang.org/grpc v1.73.0
 	google.golang.org/protobuf v1.36.6
-	k8s.io/api v0.33.1
-	k8s.io/apiextensions-apiserver v0.33.1
-	k8s.io/apimachinery v0.33.1
-	k8s.io/client-go v0.33.1
-	k8s.io/code-generator v0.33.1
-	k8s.io/component-base v0.33.1
+	k8s.io/api v0.33.2
+	k8s.io/apiextensions-apiserver v0.33.2
+	k8s.io/apimachinery v0.33.2
+	k8s.io/client-go v0.33.2
+	k8s.io/code-generator v0.33.2
+	k8s.io/component-base v0.33.2
 	k8s.io/utils v0.0.0-20241210054802-24370beab758
 	sigs.k8s.io/controller-runtime v0.21.0
 	sigs.k8s.io/gateway-api v1.3.0
@@ -108,7 +109,6 @@ require (
 	golang.org/x/mod v0.24.0 // indirect
 	golang.org/x/net v0.40.0 // indirect
 	golang.org/x/oauth2 v0.30.0 // indirect
-	golang.org/x/sync v0.14.0 // indirect
 	golang.org/x/sys v0.33.0 // indirect
 	golang.org/x/term v0.32.0 // indirect
 	golang.org/x/text v0.25.0 // indirect
@@ -122,7 +122,7 @@ require (
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
-	k8s.io/apiserver v0.33.1 // indirect
+	k8s.io/apiserver v0.33.2 // indirect
 	k8s.io/gengo/v2 v2.0.0-20250207200755-1244d31929d7 // indirect
 	k8s.io/klog/v2 v2.130.1 // indirect
 	k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff // indirect
diff --git a/go.sum b/go.sum
index 2d45d35..d4b03f7 100644
--- a/go.sum
+++ b/go.sum
@@ -162,8 +162,8 @@ github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/
 github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0=
 github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
 github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
-github.com/prometheus/common v0.64.0 h1:pdZeA+g617P7oGv1CzdTzyeShxAGrTBsolKNOLQPGO4=
-github.com/prometheus/common v0.64.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8=
+github.com/prometheus/common v0.65.0 h1:QDwzd+G1twt//Kwj/Ww6E9FQq1iVMmODnILtW1t2VzE=
+github.com/prometheus/common v0.65.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8=
 github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
 github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
 github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
@@ -295,20 +295,20 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-k8s.io/api v0.33.1 h1:tA6Cf3bHnLIrUK4IqEgb2v++/GYUtqiu9sRVk3iBXyw=
-k8s.io/api v0.33.1/go.mod h1:87esjTn9DRSRTD4fWMXamiXxJhpOIREjWOSjsW1kEHw=
-k8s.io/apiextensions-apiserver v0.33.1 h1:N7ccbSlRN6I2QBcXevB73PixX2dQNIW0ZRuguEE91zI=
-k8s.io/apiextensions-apiserver v0.33.1/go.mod h1:uNQ52z1A1Gu75QSa+pFK5bcXc4hq7lpOXbweZgi4dqA=
-k8s.io/apimachinery v0.33.1 h1:mzqXWV8tW9Rw4VeW9rEkqvnxj59k1ezDUl20tFK/oM4=
-k8s.io/apimachinery v0.33.1/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
-k8s.io/apiserver v0.33.1 h1:yLgLUPDVC6tHbNcw5uE9mo1T6ELhJj7B0geifra3Qdo=
-k8s.io/apiserver v0.33.1/go.mod h1:VMbE4ArWYLO01omz+k8hFjAdYfc3GVAYPrhP2tTKccs=
-k8s.io/client-go v0.33.1 h1:ZZV/Ks2g92cyxWkRRnfUDsnhNn28eFpt26aGc8KbXF4=
-k8s.io/client-go v0.33.1/go.mod h1:JAsUrl1ArO7uRVFWfcj6kOomSlCv+JpvIsp6usAGefA=
-k8s.io/code-generator v0.33.1 h1:ZLzIRdMsh3Myfnx9BaooX6iQry29UJjVfVG+BuS+UMw=
-k8s.io/code-generator v0.33.1/go.mod h1:HUKT7Ubp6bOgIbbaPIs9lpd2Q02uqkMCMx9/GjDrWpY=
-k8s.io/component-base v0.33.1 h1:EoJ0xA+wr77T+G8p6T3l4efT2oNwbqBVKR71E0tBIaI=
-k8s.io/component-base v0.33.1/go.mod h1:guT/w/6piyPfTgq7gfvgetyXMIh10zuXA6cRRm3rDuY=
+k8s.io/api v0.33.2 h1:YgwIS5jKfA+BZg//OQhkJNIfie/kmRsO0BmNaVSimvY=
+k8s.io/api v0.33.2/go.mod h1:fhrbphQJSM2cXzCWgqU29xLDuks4mu7ti9vveEnpSXs=
+k8s.io/apiextensions-apiserver v0.33.2 h1:6gnkIbngnaUflR3XwE1mCefN3YS8yTD631JXQhsU6M8=
+k8s.io/apiextensions-apiserver v0.33.2/go.mod h1:IvVanieYsEHJImTKXGP6XCOjTwv2LUMos0YWc9O+QP8=
+k8s.io/apimachinery v0.33.2 h1:IHFVhqg59mb8PJWTLi8m1mAoepkUNYmptHsV+Z1m5jY=
+k8s.io/apimachinery v0.33.2/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
+k8s.io/apiserver v0.33.2 h1:KGTRbxn2wJagJowo29kKBp4TchpO1DRO3g+dB/KOJN4=
+k8s.io/apiserver v0.33.2/go.mod h1:9qday04wEAMLPWWo9AwqCZSiIn3OYSZacDyu/AcoM/M=
+k8s.io/client-go v0.33.2 h1:z8CIcc0P581x/J1ZYf4CNzRKxRvQAwoAolYPbtQes+E=
+k8s.io/client-go v0.33.2/go.mod h1:9mCgT4wROvL948w6f6ArJNb7yQd7QsvqavDeZHvNmHo=
+k8s.io/code-generator v0.33.2 h1:PCJ0Y6viTCxxJHMOyGqYwWEteM4q6y1Hqo2rNpl6jF4=
+k8s.io/code-generator v0.33.2/go.mod h1:hBjCA9kPMpjLWwxcr75ReaQfFXY8u+9bEJJ7kRw3J8c=
+k8s.io/component-base v0.33.2 h1:sCCsn9s/dG3ZrQTX/Us0/Sx2R0G5kwa0wbZFYoVp/+0=
+k8s.io/component-base v0.33.2/go.mod h1:/41uw9wKzuelhN+u+/C59ixxf4tYQKW7p32ddkYNe2k=
 k8s.io/gengo/v2 v2.0.0-20250207200755-1244d31929d7 h1:2OX19X59HxDprNCVrWi6jb7LW1PoqTlYqEq5H2oetog=
 k8s.io/gengo/v2 v2.0.0-20250207200755-1244d31929d7/go.mod h1:EJykeLsmFC60UQbYJezXkEsG2FLrt0GPNkU5iK5GWxU=
 k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
diff --git a/hack/release-quickstart.sh b/hack/release-quickstart.sh
index b2f5071..068be7f 100755
--- a/hack/release-quickstart.sh
+++ b/hack/release-quickstart.sh
@@ -29,11 +29,18 @@ else
   RELEASE_TAG="v${MAJOR}.${MINOR}.0-rc.${RC}"
 fi
 
-# vLLM image version (default to 0.7.2 if not defined)
-VLLM="${VLLM:-0.7.2}"
+# The vLLM image versions
+# The GPU image is from https://hub.docker.com/layers/vllm/vllm-openai
+VLLM_GPU="${VLLM_GPU:-0.9.1}"
+# The CPU image is from https://gallery.ecr.aws/q9t5s3a7/vllm-cpu-release-repo
+VLLM_CPU="${VLLM_CPU:-0.9.1}"
+# The sim image is from https://github.com/llm-d/llm-d-inference-sim/pkgs/container/llm-d-inference-sim
+VLLM_SIM="${VLLM_SIM:-0.1.1}"
 
 echo "Using release tag: ${RELEASE_TAG}"
-echo "Using vLLM image version: ${VLLM}"
+echo "Using vLLM GPU image version: ${VLLM_GPU}"
+echo "Using vLLM CPU image version: ${VLLM_CPU}"
+echo "Using vLLM Simulator image version: ${VLLM_SIM}"
 
 # -----------------------------------------------------------------------------
 # Update pkg/README.md
@@ -64,7 +71,7 @@ sed -i.bak -E "s|(tag: )[^\"[:space:]]+|\1${RELEASE_TAG}|g" "$EPP_HELM"
 sed -i.bak -E "s|(tag: )[^\"[:space:]]+|\1${RELEASE_TAG}|g" "$BBR_HELM"
 
 # Update the container image pull policy.
-sed -i.bak '/us-central1-docker.pkg.dev\/k8s-staging-images\/gateway-api-inference-extension\/epp/ { n; s/Always/IfNotPresent/ }' "$EPP"
+sed -i.bak '/us-central1-docker.pkg.dev\/k8s-staging-images\/gateway-api-inference-extension\/epp/{n;s/Always/IfNotPresent/;}' "$EPP"
 
 # Update the container registry.
 sed -i.bak -E "s|us-central1-docker\.pkg\.dev/k8s-staging-images|registry.k8s.io|g" "$EPP"
@@ -72,22 +79,40 @@ sed -i.bak -E "s|us-central1-docker\.pkg\.dev/k8s-staging-images|registry.k8s.io
 sed -i.bak -E "s|us-central1-docker\.pkg\.dev/k8s-staging-images|registry.k8s.io|g" "$BBR_HELM"
 
 # -----------------------------------------------------------------------------
-# Update config/manifests/vllm/gpu-deployment.yaml
+# Update vLLM deployment manifests
 # -----------------------------------------------------------------------------
-VLLM_DEPLOY="config/manifests/vllm/gpu-deployment.yaml"
-echo "Updating ${VLLM_DEPLOY} ..."
+VLLM_GPU_DEPLOY="config/manifests/vllm/gpu-deployment.yaml"
+echo "Updating ${VLLM_GPU_DEPLOY} ..."
 
-# Update the vLLM image version
-sed -i.bak -E "s|(vllm/vllm-openai:)[^\"[:space:]]+|\1v${VLLM}|g" "$VLLM_DEPLOY"
+# Update the vLLM GPU image version
+sed -i.bak -E "s|(vllm/vllm-openai:)[^\"[:space:]]+|\1v${VLLM_GPU}|g" "$VLLM_GPU_DEPLOY"
 
 # Also change the imagePullPolicy from Always to IfNotPresent on lines containing the vLLM image.
-sed -i.bak '/vllm\/vllm-openai/ { n; s/Always/IfNotPresent/ }' "$VLLM_DEPLOY"
+sed -i.bak '/vllm\/vllm-openai/{n;s/Always/IfNotPresent/;}' "$VLLM_GPU_DEPLOY"
+
+VLLM_CPU_DEPLOY="config/manifests/vllm/cpu-deployment.yaml"
+echo "Updating ${VLLM_CPU_DEPLOY} ..."
+
+# Update the vLLM CPU image version
+sed -i.bak -E "s|(q9t5s3a7/vllm-cpu-release-repo:)[^\"[:space:]]+|\1v${VLLM_CPU}|g" "$VLLM_CPU_DEPLOY"
+
+# Also change the imagePullPolicy from Always to IfNotPresent on lines containing the vLLM CPU image.
+sed -i.bak '/q9t5s3a7\/vllm-cpu-release-repo/{n;s/Always/IfNotPresent/;}' "$VLLM_CPU_DEPLOY"
+
+VLLM_SIM_DEPLOY="config/manifests/vllm/sim-deployment.yaml"
+echo "Updating ${VLLM_SIM_DEPLOY} ..."
+
+# Update the vLLM Simulator image version
+sed -i.bak -E "s|(llm-d/llm-d-inference-sim:)[^\"[:space:]]+|\1v${VLLM_SIM}|g" "$VLLM_SIM_DEPLOY"
+
+# Also change the imagePullPolicy from Always to IfNotPresent on lines containing the vLLM image.
+sed -i.bak '/llm-d\/llm-d-inference-sim/{n;s/Always/IfNotPresent/;}' "$VLLM_SIM_DEPLOY"
 
 # -----------------------------------------------------------------------------
 # Stage the changes
 # -----------------------------------------------------------------------------
-echo "Staging $README $EPP $EPP_HELM $BBR_HELM $VLLM_DEPLOY files..."
-git add $README $EPP $EPP_HELM $BBR_HELM $VLLM_DEPLOY
+echo "Staging $README $EPP $EPP_HELM $BBR_HELM $VLLM_GPU_DEPLOY $VLLM_CPU_DEPLOY $VLLM_SIM_DEPLOY files..."
+git add $README $EPP $EPP_HELM $BBR_HELM $VLLM_GPU_DEPLOY $VLLM_CPU_DEPLOY $VLLM_SIM_DEPLOY
 
 # -----------------------------------------------------------------------------
 # Cleanup backup files and finish
diff --git a/mkdocs.yml b/mkdocs.yml
index 19a54e4..c8629c1 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -52,6 +52,7 @@ nav:
     - Introduction: index.md
     - Concepts:
         API Overview: concepts/api-overview.md
+        Design Principles: concepts/design-principles.md
         Conformance: concepts/conformance.md
         Roles and Personas: concepts/roles-and-personas.md
     - Implementations: 
diff --git a/pkg/epp/common/config/configloader.go b/pkg/epp/common/config/loader/configloader.go
similarity index 59%
rename from pkg/epp/common/config/configloader.go
rename to pkg/epp/common/config/loader/configloader.go
index 7db82a0..864a3ea 100644
--- a/pkg/epp/common/config/configloader.go
+++ b/pkg/epp/common/config/loader/configloader.go
@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package config
+package loader
 
 import (
 	"errors"
@@ -25,8 +25,11 @@ import (
 	"k8s.io/apimachinery/pkg/runtime/serializer"
 	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
 
+	"sigs.k8s.io/gateway-api-inference-extension/api/config/v1alpha1"
 	configapi "sigs.k8s.io/gateway-api-inference-extension/api/config/v1alpha1"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 )
 
 var scheme = runtime.NewScheme()
@@ -62,26 +65,68 @@ func LoadConfig(configText []byte, fileName string) (*configapi.EndpointPickerCo
 	return theConfig, nil
 }
 
-func LoadPluginReferences(thePlugins []configapi.PluginSpec, handle plugins.Handle) (map[string]plugins.Plugin, error) {
-	references := map[string]plugins.Plugin{}
+func LoadPluginReferences(thePlugins []configapi.PluginSpec, handle plugins.Handle) error {
 	for _, pluginConfig := range thePlugins {
-		thePlugin, err := InstantiatePlugin(pluginConfig, handle)
+		thePlugin, err := instantiatePlugin(pluginConfig, handle)
 		if err != nil {
-			return nil, err
+			return err
 		}
-		references[pluginConfig.Name] = thePlugin
+		handle.Plugins().AddPlugin(pluginConfig.Name, thePlugin)
 	}
-	return references, nil
+	return nil
+}
+
+func LoadSchedulerConfig(configProfiles []v1alpha1.SchedulingProfile, handle plugins.Handle) (*scheduling.SchedulerConfig, error) {
+
+	var profiles = map[string]*framework.SchedulerProfile{}
+
+	for _, configProfile := range configProfiles {
+		profile := framework.SchedulerProfile{}
+
+		for _, plugin := range configProfile.Plugins {
+			var err error
+			thePlugin := handle.Plugins().Plugin(plugin.PluginRef)
+			if theScorer, ok := thePlugin.(framework.Scorer); ok {
+				if plugin.Weight == nil {
+					return nil, fmt.Errorf("scorer '%s' is missing a weight", plugin.PluginRef)
+				}
+				thePlugin = framework.NewWeightedScorer(theScorer, *plugin.Weight)
+			}
+			err = profile.AddPlugins(thePlugin)
+			if err != nil {
+				return nil, err
+			}
+		}
+		profiles[configProfile.Name] = &profile
+	}
+
+	var profileHandler framework.ProfileHandler
+	var profileHandlerName string
+
+	for pluginName, thePlugin := range handle.Plugins().GetAllPluginsWithNames() {
+		if theProfileHandler, ok := thePlugin.(framework.ProfileHandler); ok {
+			if profileHandler != nil {
+				return nil, fmt.Errorf("only one profile handler is allowed. Both %s and %s are profile handlers", profileHandlerName, pluginName)
+			}
+			profileHandler = theProfileHandler
+			profileHandlerName = pluginName
+		}
+	}
+	if profileHandler == nil {
+		return nil, errors.New("no profile handler was specified")
+	}
+
+	return scheduling.NewSchedulerConfig(profileHandler, profiles), nil
 }
 
-func InstantiatePlugin(pluginSpec configapi.PluginSpec, handle plugins.Handle) (plugins.Plugin, error) {
-	factory, ok := plugins.Registry[pluginSpec.PluginName]
+func instantiatePlugin(pluginSpec configapi.PluginSpec, handle plugins.Handle) (plugins.Plugin, error) {
+	factory, ok := plugins.Registry[pluginSpec.Type]
 	if !ok {
-		return nil, fmt.Errorf("failed to instantiate the plugin. plugin %s not found", pluginSpec.PluginName)
+		return nil, fmt.Errorf("failed to instantiate the plugin. plugin type %s not found", pluginSpec.Type)
 	}
 	thePlugin, err := factory(pluginSpec.Name, pluginSpec.Parameters, handle)
 	if err != nil {
-		return nil, fmt.Errorf("failed to instantiate the plugin %s. Error: %s", pluginSpec.PluginName, err)
+		return nil, fmt.Errorf("failed to instantiate the plugin type %s. Error: %s", pluginSpec.Type, err)
 	}
 	return thePlugin, err
 }
@@ -90,18 +135,18 @@ func validateConfiguration(theConfig *configapi.EndpointPickerConfig) error {
 	names := make(map[string]struct{})
 
 	for _, pluginConfig := range theConfig.Plugins {
-		if pluginConfig.PluginName == "" {
-			return errors.New("plugin reference definition missing a plugin name")
+		if pluginConfig.Type == "" {
+			return fmt.Errorf("plugin definition for %s is missing a type", pluginConfig.Name)
 		}
 
 		if _, ok := names[pluginConfig.Name]; ok {
-			return fmt.Errorf("the name %s has been specified for more than one plugin", pluginConfig.Name)
+			return fmt.Errorf("plugin name %s used more than once", pluginConfig.Name)
 		}
 		names[pluginConfig.Name] = struct{}{}
 
-		_, ok := plugins.Registry[pluginConfig.PluginName]
+		_, ok := plugins.Registry[pluginConfig.Type]
 		if !ok {
-			return fmt.Errorf("plugin %s is not found", pluginConfig.PluginName)
+			return fmt.Errorf("plugin type %s is not found", pluginConfig.Type)
 		}
 	}
 
diff --git a/pkg/epp/plugins/plugins.go b/pkg/epp/plugins/plugins.go
index fd0f64c..bb20f6b 100644
--- a/pkg/epp/plugins/plugins.go
+++ b/pkg/epp/plugins/plugins.go
@@ -19,10 +19,29 @@ package plugins
 // Plugin defines the interface for a plugin.
 // This interface should be embedded in all plugins across the code.
 type Plugin interface {
-	// Name returns the name of the plugin.
+	// Type returns the type of the plugin.
+	Type() string
+	// Name returns the name of this plugin instance.
 	Name() string
 }
 
-// Handle provides plugins  set of standard data and tools to work with
+// Handle provides plugins a set of standard data and tools to work with
 type Handle interface {
+	// Plugins returns the sub-handle for working with instantiated plugins
+	Plugins() HandlePlugins
+}
+
+// HandlePlugins defines a set of APIs to work with instantiated plugins
+type HandlePlugins interface {
+	// Plugin returns the named plugin instance
+	Plugin(name string) Plugin
+
+	// AddPlugin adds a plugin to the set of known plugin instances
+	AddPlugin(name string, plugin Plugin)
+
+	// GetAllPlugins returns all of the known plugins
+	GetAllPlugins() []Plugin
+
+	// GetAllPluginsWithNames returns all of the known plugins with their names
+	GetAllPluginsWithNames() map[string]Plugin
 }
diff --git a/pkg/epp/plugins/registry.go b/pkg/epp/plugins/registry.go
index d33ce99..e9ff0e8 100644
--- a/pkg/epp/plugins/registry.go
+++ b/pkg/epp/plugins/registry.go
@@ -22,12 +22,12 @@ import (
 
 // Factory is the definition of the factory functions that are used to instantiate plugins
 // specified in a configuration.
-type Factory func(name string, parameters json.RawMessage, handle Handle) (Plugin, error)
+type FactoryFunc func(name string, parameters json.RawMessage, handle Handle) (Plugin, error)
 
 // Register is a static function that can be called to register plugin factory functions.
-func Register(name string, factory Factory) {
-	Registry[name] = factory
+func Register(pluginType string, factory FactoryFunc) {
+	Registry[pluginType] = factory
 }
 
 // Registry is a mapping from plugin name to Factory function
-var Registry map[string]Factory = map[string]Factory{}
+var Registry map[string]FactoryFunc = map[string]FactoryFunc{}
diff --git a/pkg/epp/registry/registry.go b/pkg/epp/registry/registry.go
deleted file mode 100644
index 8ea49da..0000000
--- a/pkg/epp/registry/registry.go
+++ /dev/null
@@ -1,35 +0,0 @@
-/*
-Copyright 2025 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package registry
-
-import (
-	"encoding/json"
-
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
-)
-
-// Factory is the definition of the factory functions that are used to instantiate plugins
-// specified in a configuration.
-type Factory func(parameters json.RawMessage) (plugins.Plugin, error)
-
-// Register is a static function that can be called to register plugin factory functions.
-func Register(name string, factory Factory) {
-	Registry[name] = factory
-}
-
-// Registry is a mapping from plugin name to Factory function
-var Registry map[string]Factory = map[string]Factory{}
diff --git a/pkg/epp/requestcontrol/director.go b/pkg/epp/requestcontrol/director.go
index 74e3c88..5bc5ca4 100644
--- a/pkg/epp/requestcontrol/director.go
+++ b/pkg/epp/requestcontrol/director.go
@@ -41,7 +41,7 @@ import (
 
 // Scheduler defines the interface required by the Director for scheduling.
 type Scheduler interface {
-	Schedule(ctx context.Context, b *schedulingtypes.LLMRequest) (result *schedulingtypes.SchedulingResult, err error)
+	Schedule(ctx context.Context, request *schedulingtypes.LLMRequest, candidatePods []schedulingtypes.Pod) (result *schedulingtypes.SchedulingResult, err error)
 }
 
 // SaturationDetector provides a signal indicating whether the backends are considered saturated.
@@ -91,12 +91,16 @@ func (d *Director) HandleRequest(ctx context.Context, reqCtx *handlers.RequestCo
 		return reqCtx, err
 	}
 
-	// NOTE: The nil checking for the modelObject means that we DO allow passthrough currently.
-	// This might be a security risk in the future where adapters not registered in the InferenceModel
-	// are able to be requested by using their distinct name.
 	modelObj := d.datastore.ModelGet(reqCtx.Model)
 	if modelObj == nil {
-		return reqCtx, errutil.Error{Code: errutil.BadConfiguration, Msg: fmt.Sprintf("error finding a model object in InferenceModel for input %v", reqCtx.Model)}
+		logger.Info("No associated inferenceModel found, using default", "model", reqCtx.Model)
+		sheddable := v1alpha2.Sheddable
+		modelObj = &v1alpha2.InferenceModel{
+			Spec: v1alpha2.InferenceModelSpec{
+				ModelName:   reqCtx.Model,
+				Criticality: &sheddable,
+			},
+		}
 	}
 
 	reqCtx.ResolvedTargetModel = reqCtx.Model
@@ -131,7 +135,11 @@ func (d *Director) HandleRequest(ctx context.Context, reqCtx *handlers.RequestCo
 	}
 
 	// --- 3. Call Scheduler ---
-	results, err := d.scheduler.Schedule(ctx, reqCtx.SchedulingRequest)
+	// Snapshot pod metrics from the datastore to:
+	// 1. Reduce concurrent access to the datastore.
+	// 2. Ensure consistent data during the scheduling operation of a request between all scheduling cycles.
+	candidatePods := schedulingtypes.ToSchedulerPodMetrics(d.datastore.PodGetAll())
+	results, err := d.scheduler.Schedule(ctx, reqCtx.SchedulingRequest, candidatePods)
 	if err != nil {
 		return reqCtx, errutil.Error{Code: errutil.InferencePoolResourceExhausted, Msg: fmt.Errorf("failed to find target pod: %w", err).Error()}
 	}
@@ -250,18 +258,18 @@ func RandomWeightedDraw(logger logr.Logger, model *v1alpha2.InferenceModel, seed
 func (d *Director) runPreRequestPlugins(ctx context.Context, request *schedulingtypes.LLMRequest, schedulingResult *schedulingtypes.SchedulingResult,
 	targetPort int) {
 	for _, plugin := range d.preRequestPlugins {
-		log.FromContext(ctx).V(logutil.DEBUG).Info("Running pre-request plugin", "plugin", plugin.Name())
+		log.FromContext(ctx).V(logutil.DEBUG).Info("Running pre-request plugin", "plugin", plugin.Type())
 		before := time.Now()
 		plugin.PreRequest(ctx, request, schedulingResult, targetPort)
-		metrics.RecordRequestControlPluginProcessingLatency(PreRequestPluginType, plugin.Name(), time.Since(before))
+		metrics.RecordRequestControlPluginProcessingLatency(PreRequestPluginType, plugin.Type(), time.Since(before))
 	}
 }
 
 func (d *Director) runPostResponsePlugins(ctx context.Context, request *schedulingtypes.LLMRequest, response *Response, targetPod *backend.Pod) {
 	for _, plugin := range d.postResponsePlugins {
-		log.FromContext(ctx).V(logutil.DEBUG).Info("Running post-response plugin", "plugin", plugin.Name())
+		log.FromContext(ctx).V(logutil.DEBUG).Info("Running post-response plugin", "plugin", plugin.Type())
 		before := time.Now()
 		plugin.PostResponse(ctx, request, response, targetPod)
-		metrics.RecordRequestControlPluginProcessingLatency(PostResponsePluginType, plugin.Name(), time.Since(before))
+		metrics.RecordRequestControlPluginProcessingLatency(PostResponsePluginType, plugin.Type(), time.Since(before))
 	}
 }
diff --git a/pkg/epp/requestcontrol/request_control_config.go b/pkg/epp/requestcontrol/request_control_config.go
index 9051b36..d658cde 100644
--- a/pkg/epp/requestcontrol/request_control_config.go
+++ b/pkg/epp/requestcontrol/request_control_config.go
@@ -16,7 +16,9 @@ limitations under the License.
 
 package requestcontrol
 
-import "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+import (
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+)
 
 // NewConfig creates a new Config object and returns its pointer.
 func NewConfig() *Config {
@@ -46,10 +48,22 @@ func (c *Config) WithPostResponsePlugins(plugins ...PostResponse) *Config {
 	return c
 }
 
-func (c *Config) AddPlugins(instances map[string]plugins.Plugin) {
-	for _, plugin := range instances {
-		if postResponse, ok := plugin.(PostResponse); ok {
-			c.postResponsePlugins = append(c.postResponsePlugins, postResponse)
+func (c *Config) AddPlugins(pluginObjects ...plugins.Plugin) {
+	for _, plugin := range pluginObjects {
+		if preRequestPlugin, ok := plugin.(PreRequest); ok {
+			c.preRequestPlugins = append(c.preRequestPlugins, preRequestPlugin)
+		}
+		if postResponsePlugin, ok := plugin.(PostResponse); ok {
+			c.postResponsePlugins = append(c.postResponsePlugins, postResponsePlugin)
 		}
 	}
 }
+
+func LoadRequestControlConfig(instantiatedPlugins map[string]plugins.Plugin) *Config {
+	config := NewConfig()
+	for _, plugin := range instantiatedPlugins {
+		config.AddPlugins(plugin)
+	}
+
+	return config
+}
diff --git a/pkg/epp/scheduling/framework/plugins.go b/pkg/epp/scheduling/framework/plugins.go
index 3d99a61..7e22d86 100644
--- a/pkg/epp/scheduling/framework/plugins.go
+++ b/pkg/epp/scheduling/framework/plugins.go
@@ -38,25 +38,28 @@ type ProfileHandler interface {
 	plugins.Plugin
 	// Pick selects the SchedulingProfiles to run from a list of candidate profiles, while taking into consideration the request properties
 	// and the previously executed SchedluderProfile cycles along with their results.
-	Pick(ctx context.Context, request *types.LLMRequest, profiles map[string]*SchedulerProfile, profileResults map[string]*types.ProfileRunResult) map[string]*SchedulerProfile
+	Pick(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest, profiles map[string]*SchedulerProfile,
+		profileResults map[string]*types.ProfileRunResult) map[string]*SchedulerProfile
 
-	// ProcessResults handles the outcome of the profile runs after all profiles ran succuessfully.
+	// ProcessResults handles the outcome of the profile runs after all profiles ran.
 	// It may aggregate results, log test profile outputs, or apply custom logic. It specifies in the SchedulingResult the
 	// key of the primary profile that should be used to get the request selected destination.
-	ProcessResults(ctx context.Context, request *types.LLMRequest, profileResults map[string]*types.ProfileRunResult) *types.SchedulingResult
+	// When a profile run fails, its result in the profileResults map is nil.
+	ProcessResults(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest,
+		profileResults map[string]*types.ProfileRunResult) (*types.SchedulingResult, error)
 }
 
 // Filter defines the interface for filtering a list of pods based on context.
 type Filter interface {
 	plugins.Plugin
-	Filter(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, pods []types.Pod) []types.Pod
+	Filter(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest, pods []types.Pod) []types.Pod
 }
 
 // Scorer defines the interface for scoring a list of pods based on context.
 // Scorers must score pods with a value within the range of [0,1] where 1 is the highest score.
 type Scorer interface {
 	plugins.Plugin
-	Score(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, pods []types.Pod) map[types.Pod]float64
+	Score(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest, pods []types.Pod) map[types.Pod]float64
 }
 
 // Picker picks the final pod(s) to send the request to.
diff --git a/pkg/epp/scheduling/framework/plugins/filter/decision_tree_filter.go b/pkg/epp/scheduling/framework/plugins/filter/decision_tree_filter.go
index 194c17b..9d3d916 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/decision_tree_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/decision_tree_filter.go
@@ -47,18 +47,26 @@ type DecisionTreeFilter struct {
 	NextOnSuccessOrFailure framework.Filter
 }
 
+// Type returns the type of the filter.
+func (f *DecisionTreeFilter) Type() string {
+	if f == nil {
+		return "nil"
+	}
+	return f.Current.Type()
+}
+
 // Name returns the name of the filter.
 func (f *DecisionTreeFilter) Name() string {
 	if f == nil {
-		return "nil"
+		return ""
 	}
 	return f.Current.Name()
 }
 
 // Filter filters out pods that doesn't meet the filter criteria.
-func (f *DecisionTreeFilter) Filter(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *DecisionTreeFilter) Filter(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest, pods []types.Pod) []types.Pod {
 	loggerTrace := log.FromContext(ctx).V(logutil.TRACE)
-	filteredPod := f.Current.Filter(ctx, request, cycleState, pods)
+	filteredPod := f.Current.Filter(ctx, cycleState, request, pods)
 
 	next := f.NextOnSuccessOrFailure
 	if len(filteredPod) > 0 {
@@ -69,9 +77,9 @@ func (f *DecisionTreeFilter) Filter(ctx context.Context, request *types.LLMReque
 		if f.NextOnSuccess != nil {
 			next = f.NextOnSuccess
 		}
-		loggerTrace.Info("Filter succeeded", "filter", f.Name(), "next", next.Name(), "filteredPodCount", len(filteredPod))
+		loggerTrace.Info("Filter succeeded", "filter", f.Type(), "next", next.Type(), "filteredPodCount", len(filteredPod))
 		// On success, pass the filtered result to the next filter.
-		return next.Filter(ctx, request, cycleState, filteredPod)
+		return next.Filter(ctx, cycleState, request, filteredPod)
 	} else {
 		if f.NextOnFailure == nil && f.NextOnSuccessOrFailure == nil {
 			// No succeeding filters to run, return.
@@ -80,8 +88,8 @@ func (f *DecisionTreeFilter) Filter(ctx context.Context, request *types.LLMReque
 		if f.NextOnFailure != nil {
 			next = f.NextOnFailure
 		}
-		loggerTrace.Info("Filter failed", "filter", f.Name(), "next", next.Name())
+		loggerTrace.Info("Filter failed", "filter", f.Type(), "next", next.Type())
 		// On failure, pass the initial set of pods to the next filter.
-		return next.Filter(ctx, request, cycleState, pods)
+		return next.Filter(ctx, cycleState, request, pods)
 	}
 }
diff --git a/pkg/epp/scheduling/framework/plugins/filter/least_kvcache_filter.go b/pkg/epp/scheduling/framework/plugins/filter/least_kvcache_filter.go
index c8f34ea..ffb190c 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/least_kvcache_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/least_kvcache_filter.go
@@ -26,19 +26,23 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
-const LeastKVCacheFilterName = "least-KV-cache"
+const (
+	LeastKVCacheFilterType = "least-KV-cache"
+)
 
 // compile-time type validation
 var _ framework.Filter = &LeastKVCacheFilter{}
 
-// LeastKVCacheFilterFactory is the plugin factory function for the Least KV Cache filter
+// LeastKVCacheFilterFactory defines the factory function for LeastKVCacheFilter.
 func LeastKVCacheFilterFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return NewLeastKVCacheFilter(), nil
+	return NewLeastKVCacheFilter().WithName(name), nil
 }
 
 // NewLeastKVCacheFilter initializes a new LeastKVCacheFilter and returns its pointer.
 func NewLeastKVCacheFilter() *LeastKVCacheFilter {
-	return &LeastKVCacheFilter{}
+	return &LeastKVCacheFilter{
+		name: LeastKVCacheFilterType,
+	}
 }
 
 // LeastKVCacheFilter finds the max and min KV cache of all pods, divides the whole range
@@ -46,15 +50,28 @@ func NewLeastKVCacheFilter() *LeastKVCacheFilter {
 // The intuition is that if there are multiple pods that share similar KV cache in the low range, we
 // should consider them all instead of the absolute minimum one. This worked better than picking the
 // least one as it gives more choices for the next filter, which on aggregate gave better results.
-type LeastKVCacheFilter struct{}
+type LeastKVCacheFilter struct {
+	name string
+}
+
+// Type returns the type of the filter.
+func (f *LeastKVCacheFilter) Type() string {
+	return LeastKVCacheFilterType
+}
 
 // Name returns the name of the filter.
 func (f *LeastKVCacheFilter) Name() string {
-	return LeastKVCacheFilterName
+	return f.name
+}
+
+// WithName sets the name of the filter.
+func (f *LeastKVCacheFilter) WithName(name string) *LeastKVCacheFilter {
+	f.name = name
+	return f
 }
 
 // Filter filters out pods that doesn't meet the filter criteria.
-func (f *LeastKVCacheFilter) Filter(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *LeastKVCacheFilter) Filter(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) []types.Pod {
 	filteredPods := []types.Pod{}
 
 	min := math.MaxFloat64
diff --git a/pkg/epp/scheduling/framework/plugins/filter/least_queue_filter.go b/pkg/epp/scheduling/framework/plugins/filter/least_queue_filter.go
index 1a30d12..54b619c 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/least_queue_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/least_queue_filter.go
@@ -26,19 +26,23 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
-const LeastQueueFilterName = "least-queue"
+const (
+	LeastQueueFilterType = "least-queue"
+)
 
 // compile-time type validation
 var _ framework.Filter = &LeastQueueFilter{}
 
-// LeastQueueFilterFactory is the plugin factory function for the Least Queue filter
+// LeastQueueFilterFactory defines the factory function for LeastQueueFilter.
 func LeastQueueFilterFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return NewLeastQueueFilter(), nil
+	return NewLeastQueueFilter().WithName(name), nil
 }
 
 // NewLeastQueueFilter initializes a new LeastQueueFilter and returns its pointer.
 func NewLeastQueueFilter() *LeastQueueFilter {
-	return &LeastQueueFilter{}
+	return &LeastQueueFilter{
+		name: LeastQueueFilterType,
+	}
 }
 
 // LeastQueueFilter finds the max and min queue size of all pods, divides the whole range
@@ -46,15 +50,28 @@ func NewLeastQueueFilter() *LeastQueueFilter {
 // The intuition is that if there are multiple pods that share similar queue size in the low range,
 // we should consider them all instead of the absolute minimum one. This worked better than picking
 // the least one as it gives more choices for the next filter, which on aggregate gave better results.
-type LeastQueueFilter struct{}
+type LeastQueueFilter struct {
+	name string
+}
+
+// Type returns the type of the filter.
+func (f *LeastQueueFilter) Type() string {
+	return LeastQueueFilterType
+}
 
 // Name returns the name of the filter.
 func (f *LeastQueueFilter) Name() string {
-	return LeastQueueFilterName
+	return f.name
+}
+
+// WithName sets the name of the filter.
+func (f *LeastQueueFilter) WithName(name string) *LeastQueueFilter {
+	f.name = name
+	return f
 }
 
 // Filter filters out pods that doesn't meet the filter criteria.
-func (f *LeastQueueFilter) Filter(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *LeastQueueFilter) Filter(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) []types.Pod {
 	filteredPods := []types.Pod{}
 
 	min := math.MaxInt
diff --git a/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go b/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go
index 1d89e48..edb18a9 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go
@@ -29,7 +29,9 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
-const LoraAffinityFilterName = "lora-affinity"
+const (
+	LoraAffinityFilterType = "lora-affinity"
+)
 
 type loraAffinityFilterParameters struct {
 	Threshold float64 `json:"threshold"`
@@ -38,19 +40,20 @@ type loraAffinityFilterParameters struct {
 // compile-time type validation
 var _ framework.Filter = &LoraAffinityFilter{}
 
-// LoraAffinityFilterFactory is the factory function for the LoraAffinity filter
+// LoraAffinityFilterFactory defines the factory function for LoraAffinityFilter.
 func LoraAffinityFilterFactory(name string, rawParameters json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
 	parameters := loraAffinityFilterParameters{Threshold: config.DefaultLoraAffinityThreshold}
 	if err := json.Unmarshal(rawParameters, &parameters); err != nil {
-		return nil, fmt.Errorf("failed to parse the parameters of the %s filter. Error: %s", LoraAffinityFilterName, err)
+		return nil, fmt.Errorf("failed to parse the parameters of the '%s' filter - %w", LoraAffinityFilterType, err)
 	}
-	return &LoraAffinityFilter{loraAffinityThreshold: parameters.Threshold}, nil
+	return NewLoraAffinityFilter(parameters.Threshold).WithName(name), nil
 }
 
 // NewLoraAffinityFilter initializes a new LoraAffinityFilter and returns its pointer.
-func NewLoraAffinityFilter() *LoraAffinityFilter {
+func NewLoraAffinityFilter(threshold float64) *LoraAffinityFilter {
 	return &LoraAffinityFilter{
-		loraAffinityThreshold: config.Conf.LoraAffinityThreshold,
+		name:                  LoraAffinityFilterType,
+		loraAffinityThreshold: threshold,
 	}
 }
 
@@ -62,16 +65,28 @@ func NewLoraAffinityFilter() *LoraAffinityFilter {
 // 2. Using a probability threshold to sometimes select from non-affinity pods to enable load balancing
 // 3. Falling back to whatever group has pods if one group is empty
 type LoraAffinityFilter struct {
+	name                  string
 	loraAffinityThreshold float64
 }
 
-// Name returns the name of the filter.
+// Type returns the type of the filter.
+func (f *LoraAffinityFilter) Type() string {
+	return LoraAffinityFilterType
+}
+
+// Name returns the type of the filter.
 func (f *LoraAffinityFilter) Name() string {
-	return LoraAffinityFilterName
+	return f.name
+}
+
+// WithName sets the type of the filter.
+func (f *LoraAffinityFilter) WithName(name string) *LoraAffinityFilter {
+	f.name = name
+	return f
 }
 
 // Filter filters out pods that doesn't meet the filter criteria.
-func (f *LoraAffinityFilter) Filter(_ context.Context, request *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *LoraAffinityFilter) Filter(_ context.Context, _ *types.CycleState, request *types.LLMRequest, pods []types.Pod) []types.Pod {
 	// Pre-allocate slices with estimated capacity
 	filtered_affinity := make([]types.Pod, 0, len(pods))
 	filtered_available := make([]types.Pod, 0, len(pods))
diff --git a/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go b/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go
index 9ab0717..b72ccf8 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go
@@ -28,7 +28,9 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
-const LowQueueFilterName = "low-queue"
+const (
+	LowQueueFilterType = "low-queue"
+)
 
 type lowQueueFilterParameters struct {
 	Threshold int `json:"threshold"`
@@ -37,35 +39,48 @@ type lowQueueFilterParameters struct {
 // compile-time type validation
 var _ framework.Filter = &LowQueueFilter{}
 
-// LowQueueFilterFactory is the factory function for the LowQueue filter
+// LowQueueFilterFactory defines the factory function for LowQueueFilter.
 func LowQueueFilterFactory(name string, rawParameters json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
 	parameters := lowQueueFilterParameters{Threshold: config.DefaultQueueingThresholdLoRA}
 	if err := json.Unmarshal(rawParameters, &parameters); err != nil {
-		return nil, fmt.Errorf("failed to parse the parameters of the %s filter. Error: %s", LowQueueFilterName, err)
+		return nil, fmt.Errorf("failed to parse the parameters of the '%s' filter - %w", LowQueueFilterType, err)
 	}
 
-	return &LowQueueFilter{queueingThresholdLoRA: parameters.Threshold}, nil
+	return NewLowQueueFilter(parameters.Threshold).WithName(name), nil
 }
 
 // NewLowQueueFilter initializes a new LowQueueFilter and returns its pointer.
-func NewLowQueueFilter() *LowQueueFilter {
+func NewLowQueueFilter(threshold int) *LowQueueFilter {
 	return &LowQueueFilter{
-		queueingThresholdLoRA: config.Conf.QueueingThresholdLoRA,
+		name:                  LowQueueFilterType,
+		queueingThresholdLoRA: threshold,
 	}
 }
 
 // LowQueueFilter returns pods that their waiting queue size is less than a configured threshold
 type LowQueueFilter struct {
+	name                  string
 	queueingThresholdLoRA int
 }
 
+// Type returns the type of the filter.
+func (f *LowQueueFilter) Type() string {
+	return LowQueueFilterType
+}
+
 // Name returns the name of the filter.
 func (f *LowQueueFilter) Name() string {
-	return LowQueueFilterName
+	return f.name
+}
+
+// WithName sets the name of the filter.
+func (f *LowQueueFilter) WithName(name string) *LowQueueFilter {
+	f.name = name
+	return f
 }
 
 // Filter filters out pods that doesn't meet the filter criteria.
-func (f *LowQueueFilter) Filter(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *LowQueueFilter) Filter(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) []types.Pod {
 	filteredPods := []types.Pod{}
 
 	for _, pod := range pods {
diff --git a/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin.go b/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin.go
index 6ee8253..a4656a6 100644
--- a/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin.go
+++ b/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin.go
@@ -52,7 +52,7 @@ const (
 	// in vLLM, we will have 250K / 16 = 31.25K blocks.
 	DefaultLRUCapacityPerServer = 31250
 
-	PrefixCachePluginName = "prefix-cache"
+	PrefixCachePluginType = "prefix-cache"
 )
 
 type Config struct {
@@ -68,6 +68,7 @@ type Config struct {
 
 type Plugin struct {
 	Config
+	name    string
 	indexer Indexer
 }
 
@@ -88,23 +89,6 @@ func (s ServerID) String() string {
 	return k8stypes.NamespacedName(s).String()
 }
 
-// PrefixCachePluginFactory is the factory for the PrefixCache plugin
-func PrefixCachePluginFactory(name string, rawParameters json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	parameters := Config{
-		HashBlockSize:          DefaultHashBlockSize,
-		MaxPrefixBlocksToMatch: DefaultMaxPrefixBlocks,
-		LRUCapacityPerServer:   DefaultLRUCapacityPerServer,
-	}
-	if err := json.Unmarshal(rawParameters, &parameters); err != nil {
-		return nil, fmt.Errorf("failed to parse the parameters of the %s plugin. Error: %s", PrefixCachePluginName, err)
-	}
-
-	return &Plugin{
-		Config:  parameters,
-		indexer: newIndexer(parameters.LRUCapacityPerServer),
-	}, nil
-}
-
 // compile-time type validation
 var _ types.StateData = &schedulingContextState{}
 
@@ -134,6 +118,20 @@ func (s *schedulingContextState) Clone() types.StateData {
 var _ framework.Scorer = &Plugin{}
 var _ framework.PostCycle = &Plugin{}
 
+// PrefixCachePluginFactory defines the factory function for Prefix plugin.
+func PrefixCachePluginFactory(name string, rawParameters json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
+	parameters := Config{
+		HashBlockSize:          DefaultHashBlockSize,
+		MaxPrefixBlocksToMatch: DefaultMaxPrefixBlocks,
+		LRUCapacityPerServer:   DefaultLRUCapacityPerServer,
+	}
+	if err := json.Unmarshal(rawParameters, &parameters); err != nil {
+		return nil, fmt.Errorf("failed to parse the parameters of the %s plugin. Error: %s", PrefixCachePluginType, err)
+	}
+
+	return New(parameters).WithName(name), nil
+}
+
 // New initializes a new prefix Plugin and returns its pointer.
 func New(config Config) *Plugin {
 	capacity := config.LRUCapacityPerServer
@@ -145,20 +143,31 @@ func New(config Config) *Plugin {
 		)
 	}
 
-	m := &Plugin{
+	return &Plugin{
+		name:    PrefixCachePluginType,
 		Config:  config,
 		indexer: newIndexer(capacity),
 	}
-	return m
+}
+
+// Type returns the type of the plugin.
+func (m *Plugin) Type() string {
+	return PrefixCachePluginType
 }
 
 // Name returns the name of the plugin.
 func (m *Plugin) Name() string {
-	return PrefixCachePluginName
+	return m.name
+}
+
+// WithName sets the name of the plugin.
+func (m *Plugin) WithName(name string) *Plugin {
+	m.name = name
+	return m
 }
 
 // Score returns the scoring result for the given list of pods based on context.
-func (m *Plugin) Score(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, pods []types.Pod) map[types.Pod]float64 {
+func (m *Plugin) Score(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest, pods []types.Pod) map[types.Pod]float64 {
 	loggerTrace := log.FromContext(ctx).V(logutil.TRACE)
 	// pre score step, hashing prompt and find longest prefix match.
 	hashes := hashPrompt(ctx, request, m.HashBlockSize, m.MaxPrefixBlocksToMatch)
@@ -167,7 +176,7 @@ func (m *Plugin) Score(ctx context.Context, request *types.LLMRequest, cycleStat
 		PrefixCacheServers: m.matchLongestPrefix(ctx, hashes),
 	}
 
-	cycleState.Write(types.StateKey(m.Name()), state)
+	cycleState.Write(types.StateKey(m.Type()), state)
 	loggerTrace.Info(fmt.Sprintf("cached servers: %+v", state.PrefixCacheServers), "hashes", state.PrefixHashes)
 	// calculate the scores of pods
 	scores := make(map[types.Pod]float64, len(pods))
@@ -228,7 +237,7 @@ func (m *Plugin) matchLongestPrefix(ctx context.Context, hashes []BlockHash) map
 
 // getPrefixState returns the cycle state as a schedulingContextState.
 func (m *Plugin) getPrefixState(cycleState *types.CycleState) (*schedulingContextState, error) {
-	prefixStateKey := types.StateKey(m.Name())
+	prefixStateKey := types.StateKey(m.Type())
 	state, err := cycleState.Read(prefixStateKey)
 	if err != nil {
 		return nil, fmt.Errorf("failed reading %q from CycleState: %w", prefixStateKey, err)
diff --git a/pkg/epp/scheduling/framework/plugins/picker/max_score_picker.go b/pkg/epp/scheduling/framework/plugins/picker/max_score_picker.go
index f358c16..db4e848 100644
--- a/pkg/epp/scheduling/framework/plugins/picker/max_score_picker.go
+++ b/pkg/epp/scheduling/framework/plugins/picker/max_score_picker.go
@@ -28,31 +28,46 @@ import (
 	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
 )
 
-const MaxScorePickerName = "max-score"
+const (
+	MaxScorePickerType = "max-score"
+)
 
 // compile-time type validation
 var _ framework.Picker = &MaxScorePicker{}
 
-// MaxScorePickerFactory is the factory for the MaxScore picker
+// MaxScorePickerFactory defines the factory function for MaxScorePicker.
 func MaxScorePickerFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return &MaxScorePicker{random: NewRandomPicker()}, nil
+	return NewMaxScorePicker().WithName(name), nil
 }
 
 // NewMaxScorePicker initializes a new MaxScorePicker and returns its pointer.
 func NewMaxScorePicker() *MaxScorePicker {
 	return &MaxScorePicker{
+		name:   MaxScorePickerType,
 		random: NewRandomPicker(),
 	}
 }
 
 // MaxScorePicker picks the pod with the maximum score from the list of candidates.
 type MaxScorePicker struct {
+	name   string
 	random *RandomPicker
 }
 
+// Type returns the type of the picker.
+func (p *MaxScorePicker) Type() string {
+	return MaxScorePickerType
+}
+
 // Name returns the name of the picker.
 func (p *MaxScorePicker) Name() string {
-	return MaxScorePickerName
+	return p.name
+}
+
+// WithName sets the picker's name
+func (p *MaxScorePicker) WithName(name string) *MaxScorePicker {
+	p.name = name
+	return p
 }
 
 // Pick selects the pod with the maximum score from the list of candidates.
diff --git a/pkg/epp/scheduling/framework/plugins/picker/random_picker.go b/pkg/epp/scheduling/framework/plugins/picker/random_picker.go
index 8dec5e2..fb2d44b 100644
--- a/pkg/epp/scheduling/framework/plugins/picker/random_picker.go
+++ b/pkg/epp/scheduling/framework/plugins/picker/random_picker.go
@@ -29,27 +29,44 @@ import (
 	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
 )
 
-const RandomPickerName = "random"
+const (
+	RandomPickerType = "random"
+)
 
 // compile-time type validation
 var _ framework.Picker = &RandomPicker{}
 
-// RandomPickerFactory is the factory for the Random picker
+// RandomPickerFactory defines the factory function for RandomPicker.
 func RandomPickerFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return NewRandomPicker(), nil
+	return NewRandomPicker().WithName(name), nil
 }
 
 // NewRandomPicker initializes a new RandomPicker and returns its pointer.
 func NewRandomPicker() *RandomPicker {
-	return &RandomPicker{}
+	return &RandomPicker{
+		name: RandomPickerType,
+	}
 }
 
 // RandomPicker picks a random pod from the list of candidates.
-type RandomPicker struct{}
+type RandomPicker struct {
+	name string
+}
+
+// Type returns the type of the picker.
+func (p *RandomPicker) Type() string {
+	return RandomPickerType
+}
 
 // Name returns the name of the picker.
 func (p *RandomPicker) Name() string {
-	return RandomPickerName
+	return p.name
+}
+
+// WithName sets the picker's name
+func (p *RandomPicker) WithName(name string) *RandomPicker {
+	p.name = name
+	return p
 }
 
 // Pick selects a random pod from the list of candidates.
diff --git a/pkg/epp/scheduling/framework/plugins/profile/single_profile_handler.go b/pkg/epp/scheduling/framework/plugins/profile/single_profile_handler.go
index f31811a..48d75a6 100644
--- a/pkg/epp/scheduling/framework/plugins/profile/single_profile_handler.go
+++ b/pkg/epp/scheduling/framework/plugins/profile/single_profile_handler.go
@@ -19,37 +19,57 @@ package profile
 import (
 	"context"
 	"encoding/json"
+	"errors"
+	"fmt"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
-const SingleProfileHandlerName = "single-profile"
+const (
+	SingleProfileHandlerType = "single-profile"
+)
 
 // compile-time type assertion
 var _ framework.ProfileHandler = &SingleProfileHandler{}
 
+// SingleProfileHandlerFactory defines the factory function for SingleProfileHandler.
 func SingleProfileHandlerFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return NewSingleProfileHandler(), nil
+	return NewSingleProfileHandler().WithName(name), nil
 }
 
 // NewSingleProfileHandler initializes a new SingleProfileHandler and returns its pointer.
 func NewSingleProfileHandler() *SingleProfileHandler {
-	return &SingleProfileHandler{}
+	return &SingleProfileHandler{
+		name: SingleProfileHandlerType,
+	}
 }
 
 // SingleProfileHandler handles a single profile which is always the primary profile.
-type SingleProfileHandler struct{}
+type SingleProfileHandler struct {
+	name string
+}
 
-// Name returns the name of the Profiles Picker.
+// Type returns the type of the Profile Handler.
+func (h *SingleProfileHandler) Type() string {
+	return SingleProfileHandlerType
+}
+
+// Name returns the name of the profile handler.
 func (h *SingleProfileHandler) Name() string {
-	return SingleProfileHandlerName
+	return h.name
+}
+
+// WithName sets the name of the profile handler.
+func (h *SingleProfileHandler) WithName(name string) *SingleProfileHandler {
+	h.name = name
+	return h
 }
 
 // Pick selects the SchedulingProfiles to run from the list of candidate profiles, while taking into consideration the request properties and the
 // previously executed cycles along with their results.
-func (h *SingleProfileHandler) Pick(_ context.Context, request *types.LLMRequest, profiles map[string]*framework.SchedulerProfile,
+func (h *SingleProfileHandler) Pick(_ context.Context, _ *types.CycleState, request *types.LLMRequest, profiles map[string]*framework.SchedulerProfile,
 	profileResults map[string]*types.ProfileRunResult) map[string]*framework.SchedulerProfile {
 	if len(profiles) == len(profileResults) { // all profiles have been executed already in previous call
 		return map[string]*framework.SchedulerProfile{}
@@ -58,15 +78,28 @@ func (h *SingleProfileHandler) Pick(_ context.Context, request *types.LLMRequest
 	return profiles
 }
 
-func (h *SingleProfileHandler) ProcessResults(_ context.Context, _ *types.LLMRequest, profileResults map[string]*types.ProfileRunResult) *types.SchedulingResult {
-	var firstKey string
-	for key := range profileResults {
-		firstKey = key
+// ProcessResults handles the outcome of the profile runs after all profiles ran.
+// It may aggregate results, log test profile outputs, or apply custom logic. It specifies in the SchedulingResult the
+// key of the primary profile that should be used to get the request selected destination.
+// When a profile run fails, its result in the profileResults map is nil.
+func (h *SingleProfileHandler) ProcessResults(_ context.Context, _ *types.CycleState, _ *types.LLMRequest,
+	profileResults map[string]*types.ProfileRunResult) (*types.SchedulingResult, error) {
+	if len(profileResults) != 1 {
+		return nil, errors.New("single profile handler is intended to be used with a single profile, failed to process multiple profiles")
+	}
+
+	var singleProfileName string
+	for profileName := range profileResults {
+		singleProfileName = profileName
 		break
 	}
 
+	if profileResults[singleProfileName] == nil { // there was an error while running the profile
+		return nil, fmt.Errorf("failed to run scheduler profile '%s'", singleProfileName)
+	}
+
 	return &types.SchedulingResult{
 		ProfileResults:     profileResults,
-		PrimaryProfileName: firstKey,
-	}
+		PrimaryProfileName: singleProfileName,
+	}, nil
 }
diff --git a/pkg/epp/scheduling/framework/plugins/scorer/kvcache.go b/pkg/epp/scheduling/framework/plugins/scorer/kvcache.go
index 50e6485..f0d4af9 100644
--- a/pkg/epp/scheduling/framework/plugins/scorer/kvcache.go
+++ b/pkg/epp/scheduling/framework/plugins/scorer/kvcache.go
@@ -27,32 +27,47 @@ import (
 
 const (
 	DefaultKVCacheScorerWeight = 1
-	KvCacheScorerName          = "kv-cache"
+	KvCacheScorerType          = "kv-cache"
 )
 
 // compile-time type assertion
 var _ framework.Scorer = &KVCacheScorer{}
 
-// KvCacheScorerFactory is the factory for the KV-Cache scorer
+// KvCacheScorerFactory defines the factory function for KVCacheScorer.
 func KvCacheScorerFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return &KVCacheScorer{}, nil
+	return NewKVCacheScorer().WithName(name), nil
 }
 
 // NewKVCacheScorer initializes a new KVCacheScorer and returns its pointer.
 func NewKVCacheScorer() *KVCacheScorer {
-	return &KVCacheScorer{}
+	return &KVCacheScorer{
+		name: KvCacheScorerType,
+	}
 }
 
 // KVCacheScorer scores list of candidate pods based on KV cache utilization.
-type KVCacheScorer struct{}
+type KVCacheScorer struct {
+	name string
+}
+
+// Type returns the type of the scorer.
+func (s *KVCacheScorer) Type() string {
+	return KvCacheScorerType
+}
 
 // Name returns the name of the scorer.
 func (s *KVCacheScorer) Name() string {
-	return KvCacheScorerName
+	return s.name
+}
+
+// WithName sets the name of the scorer.
+func (s *KVCacheScorer) WithName(name string) *KVCacheScorer {
+	s.name = name
+	return s
 }
 
 // Score returns the scoring result for the given list of pods based on context.
-func (s *KVCacheScorer) Score(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) map[types.Pod]float64 {
+func (s *KVCacheScorer) Score(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) map[types.Pod]float64 {
 	scores := make(map[types.Pod]float64, len(pods))
 	for _, pod := range pods {
 		scores[pod] = 1 - pod.GetMetrics().KVCacheUsagePercent
diff --git a/pkg/epp/scheduling/framework/plugins/scorer/queue.go b/pkg/epp/scheduling/framework/plugins/scorer/queue.go
index 0a769b8..dad6d04 100644
--- a/pkg/epp/scheduling/framework/plugins/scorer/queue.go
+++ b/pkg/epp/scheduling/framework/plugins/scorer/queue.go
@@ -28,33 +28,48 @@ import (
 
 const (
 	DefaultQueueScorerWeight = 1
-	QueueScorerName          = "queue"
+	QueueScorerType          = "queue"
 )
 
 // compile-time type assertion
 var _ framework.Scorer = &QueueScorer{}
 
-// QueueScorerFactory is the factory for the Queue based scorer
+// QueueScorerFactory defines the factory function for QueueScorer.
 func QueueScorerFactory(name string, _ json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	return &QueueScorer{}, nil
+	return NewQueueScorer().WithName(name), nil
 }
 
 // NewQueueScorer initializes a new QueueScorer and returns its pointer.
 func NewQueueScorer() *QueueScorer {
-	return &QueueScorer{}
+	return &QueueScorer{
+		name: QueueScorerType,
+	}
 }
 
 // QueueScorer scores list of candidate pods based on the pod's waiting queue size.
 // the less waiting queue size the pod has, the higher score it will get (since it's more available to serve new request).
-type QueueScorer struct{}
+type QueueScorer struct {
+	name string
+}
+
+// Type returns the type of the scorer.
+func (s *QueueScorer) Type() string {
+	return QueueScorerType
+}
 
 // Name returns the name of the scorer.
 func (s *QueueScorer) Name() string {
-	return QueueScorerName
+	return s.name
+}
+
+// WithName sets the name of the scorer.
+func (s *QueueScorer) WithName(name string) *QueueScorer {
+	s.name = name
+	return s
 }
 
 // Score returns the scoring result for the given list of pods based on context.
-func (s *QueueScorer) Score(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) map[types.Pod]float64 {
+func (s *QueueScorer) Score(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) map[types.Pod]float64 {
 	minQueueSize := math.MaxInt
 	maxQueueSize := math.MinInt
 
diff --git a/pkg/epp/scheduling/framework/scheduler_profile.go b/pkg/epp/scheduling/framework/scheduler_profile.go
index 994d829..f41a915 100644
--- a/pkg/epp/scheduling/framework/scheduler_profile.go
+++ b/pkg/epp/scheduling/framework/scheduler_profile.go
@@ -86,14 +86,14 @@ func (p *SchedulerProfile) AddPlugins(pluginObjects ...plugins.Plugin) error {
 			p.scorers = append(p.scorers, weightedScorer)
 			plugin = weightedScorer.Scorer // if we got WeightedScorer, unwrap the plugin
 		} else if scorer, ok := plugin.(Scorer); ok { // if we got a Scorer instead of WeightedScorer that's an error.
-			return fmt.Errorf("failed to register scorer '%s' without a weight. follow function documentation to register a scorer", scorer.Name())
+			return fmt.Errorf("failed to register scorer '%s' without a weight. follow function documentation to register a scorer", scorer.Type())
 		}
 		if filter, ok := plugin.(Filter); ok {
 			p.filters = append(p.filters, filter)
 		}
 		if picker, ok := plugin.(Picker); ok {
 			if p.picker != nil {
-				return fmt.Errorf("failed to set '%s' as picker, already have a registered picker plugin '%s'", picker.Name(), p.picker.Name())
+				return fmt.Errorf("failed to set '%s' as picker, already have a registered picker plugin '%s'", picker.Type(), p.picker.Type())
 			}
 			p.picker = picker
 		}
@@ -106,8 +106,8 @@ func (p *SchedulerProfile) AddPlugins(pluginObjects ...plugins.Plugin) error {
 
 // RunCycle runs a SchedulerProfile cycle. In other words, it invokes all the SchedulerProfile plugins in this
 // order - Filters, Scorers, Picker, PostCyclePlugins. After completing all, it returns the result.
-func (p *SchedulerProfile) Run(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, podsSnapshot []types.Pod) (*types.ProfileRunResult, error) {
-	pods := p.runFilterPlugins(ctx, request, cycleState, podsSnapshot)
+func (p *SchedulerProfile) Run(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, candidatePods []types.Pod) (*types.ProfileRunResult, error) {
+	pods := p.runFilterPlugins(ctx, request, cycleState, candidatePods)
 	if len(pods) == 0 {
 		return nil, errutil.Error{Code: errutil.Internal, Msg: "no pods available for the given request"}
 	}
@@ -127,11 +127,11 @@ func (p *SchedulerProfile) runFilterPlugins(ctx context.Context, request *types.
 	loggerDebug.Info("Before running filter plugins", "pods", filteredPods)
 
 	for _, filter := range p.filters {
-		loggerDebug.Info("Running filter plugin", "plugin", filter.Name())
+		loggerDebug.Info("Running filter plugin", "plugin", filter.Type())
 		before := time.Now()
-		filteredPods = filter.Filter(ctx, request, cycleState, filteredPods)
-		metrics.RecordSchedulerPluginProcessingLatency(FilterPluginType, filter.Name(), time.Since(before))
-		loggerDebug.Info("Filter plugin result", "plugin", filter.Name(), "pods", filteredPods)
+		filteredPods = filter.Filter(ctx, cycleState, request, filteredPods)
+		metrics.RecordSchedulerPluginProcessingLatency(FilterPluginType, filter.Type(), time.Since(before))
+		loggerDebug.Info("Filter plugin result", "plugin", filter.Type(), "pods", filteredPods)
 		if len(filteredPods) == 0 {
 			break
 		}
@@ -151,14 +151,14 @@ func (p *SchedulerProfile) runScorerPlugins(ctx context.Context, request *types.
 	}
 	// Iterate through each scorer in the chain and accumulate the weighted scores.
 	for _, scorer := range p.scorers {
-		loggerDebug.Info("Running scorer", "scorer", scorer.Name())
+		loggerDebug.Info("Running scorer", "scorer", scorer.Type())
 		before := time.Now()
-		scores := scorer.Score(ctx, request, cycleState, pods)
-		metrics.RecordSchedulerPluginProcessingLatency(ScorerPluginType, scorer.Name(), time.Since(before))
+		scores := scorer.Score(ctx, cycleState, request, pods)
+		metrics.RecordSchedulerPluginProcessingLatency(ScorerPluginType, scorer.Type(), time.Since(before))
 		for pod, score := range scores { // weight is relative to the sum of weights
 			weightedScorePerPod[pod] += score * float64(scorer.Weight())
 		}
-		loggerDebug.Info("After running scorer", "scorer", scorer.Name())
+		loggerDebug.Info("After running scorer", "scorer", scorer.Type())
 	}
 	loggerDebug.Info("After running scorer plugins")
 
@@ -177,7 +177,7 @@ func (p *SchedulerProfile) runPickerPlugin(ctx context.Context, cycleState *type
 	loggerDebug.Info("Before running picker plugin", "pods weighted score", fmt.Sprint(weightedScorePerPod))
 	before := time.Now()
 	result := p.picker.Pick(ctx, cycleState, scoredPods)
-	metrics.RecordSchedulerPluginProcessingLatency(PickerPluginType, p.picker.Name(), time.Since(before))
+	metrics.RecordSchedulerPluginProcessingLatency(PickerPluginType, p.picker.Type(), time.Since(before))
 	loggerDebug.Info("After running picker plugin", "result", result)
 
 	return result
@@ -185,9 +185,9 @@ func (p *SchedulerProfile) runPickerPlugin(ctx context.Context, cycleState *type
 
 func (p *SchedulerProfile) runPostCyclePlugins(ctx context.Context, cycleState *types.CycleState, result *types.ProfileRunResult) {
 	for _, plugin := range p.postCyclePlugins {
-		log.FromContext(ctx).V(logutil.DEBUG).Info("Running post-cycle plugin", "plugin", plugin.Name())
+		log.FromContext(ctx).V(logutil.DEBUG).Info("Running post-cycle plugin", "plugin", plugin.Type())
 		before := time.Now()
 		plugin.PostCycle(ctx, cycleState, result)
-		metrics.RecordSchedulerPluginProcessingLatency(PostCyclePluginType, plugin.Name(), time.Since(before))
+		metrics.RecordSchedulerPluginProcessingLatency(PostCyclePluginType, plugin.Type(), time.Since(before))
 	}
 }
diff --git a/pkg/epp/scheduling/scheduler.go b/pkg/epp/scheduling/scheduler.go
index 82c8e99..b848b26 100644
--- a/pkg/epp/scheduling/scheduler.go
+++ b/pkg/epp/scheduling/scheduler.go
@@ -25,6 +25,7 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/log"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/filter"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
@@ -33,17 +34,21 @@ import (
 	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
 )
 
+type Datastore interface {
+	PodGetAll() []backendmetrics.PodMetrics
+}
+
 // NewScheduler returns a new scheduler with default scheduler plugins configuration.
-func NewScheduler(datastore Datastore) *Scheduler {
+func NewScheduler() *Scheduler {
 	// When the scheduler is initialized with NewScheduler function, thw below config will be used as default.
 	// it's possible to call NewSchedulerWithConfig to pass a different scheduler config.
 	// For build time plugins changes, it's recommended to call in main.go to NewSchedulerWithConfig.
-	loraAffinityFilter := filter.NewLoraAffinityFilter()
+	loraAffinityFilter := filter.NewLoraAffinityFilter(config.Conf.LoraAffinityThreshold)
 	leastQueueFilter := filter.NewLeastQueueFilter()
 	leastKvCacheFilter := filter.NewLeastKVCacheFilter()
 
 	lowLatencyFilter := &filter.DecisionTreeFilter{
-		Current: filter.NewLowQueueFilter(),
+		Current: filter.NewLowQueueFilter(config.Conf.QueueingThresholdLoRA),
 		NextOnSuccess: &filter.DecisionTreeFilter{
 			Current: loraAffinityFilter,
 			NextOnSuccessOrFailure: &filter.DecisionTreeFilter{
@@ -70,30 +75,24 @@ func NewScheduler(datastore Datastore) *Scheduler {
 
 	profileHandler := profile.NewSingleProfileHandler()
 
-	return NewSchedulerWithConfig(datastore, NewSchedulerConfig(profileHandler, map[string]*framework.SchedulerProfile{"default": defaultProfile}))
+	return NewSchedulerWithConfig(NewSchedulerConfig(profileHandler, map[string]*framework.SchedulerProfile{"default": defaultProfile}))
 }
 
 // NewSchedulerWithConfig returns a new scheduler with the given scheduler plugins configuration.
-func NewSchedulerWithConfig(datastore Datastore, config *SchedulerConfig) *Scheduler {
+func NewSchedulerWithConfig(config *SchedulerConfig) *Scheduler {
 	return &Scheduler{
-		datastore:      datastore,
 		profileHandler: config.profileHandler,
 		profiles:       config.profiles,
 	}
 }
 
 type Scheduler struct {
-	datastore      Datastore
 	profileHandler framework.ProfileHandler
 	profiles       map[string]*framework.SchedulerProfile
 }
 
-type Datastore interface {
-	PodGetAll() []backendmetrics.PodMetrics
-}
-
 // Schedule finds the target pod based on metrics and the requested lora adapter.
-func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest) (*types.SchedulingResult, error) {
+func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest, candidatePods []types.Pod) (*types.SchedulingResult, error) {
 	logger := log.FromContext(ctx).WithValues("request", request)
 	loggerDebug := logger.V(logutil.DEBUG)
 
@@ -102,31 +101,25 @@ func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest) (*t
 		metrics.RecordSchedulerE2ELatency(time.Since(scheduleStart))
 	}()
 
-	// Snapshot pod metrics from the datastore to:
-	// 1. Reduce concurrent access to the datastore.
-	// 2. Ensure consistent data during the scheduling operation of a request between all scheduling cycles.
-	podsSnapshot := types.ToSchedulerPodMetrics(s.datastore.PodGetAll())
-	loggerDebug.Info(fmt.Sprintf("Scheduling a request, Metrics: %+v", podsSnapshot))
-
 	profileRunResults := map[string]*types.ProfileRunResult{}
 	cycleState := types.NewCycleState()
 
 	for { // get the next set of profiles to run iteratively based on the request and the previous execution results
 		before := time.Now()
-		profiles := s.profileHandler.Pick(ctx, request, s.profiles, profileRunResults)
-		metrics.RecordSchedulerPluginProcessingLatency(framework.ProfilePickerType, s.profileHandler.Name(), time.Since(before))
+		profiles := s.profileHandler.Pick(ctx, cycleState, request, s.profiles, profileRunResults)
+		metrics.RecordSchedulerPluginProcessingLatency(framework.ProfilePickerType, s.profileHandler.Type(), time.Since(before))
 		if len(profiles) == 0 { // profile picker didn't pick any profile to run
 			break
 		}
 
 		for name, profile := range profiles {
 			// run the selected profiles and collect results (current code runs all profiles)
-			profileRunResult, err := profile.Run(ctx, request, cycleState, podsSnapshot)
+			profileRunResult, err := profile.Run(ctx, request, cycleState, candidatePods)
 			if err != nil {
-				return nil, fmt.Errorf("failed to run all required scheduling profiles - %w", err)
+				loggerDebug.Info("failed to run scheduler profile", "profile", name, "error", err.Error())
 			}
 
-			profileRunResults[name] = profileRunResult
+			profileRunResults[name] = profileRunResult // if profile failed to run, the run result is nil
 		}
 	}
 
@@ -135,8 +128,8 @@ func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest) (*t
 	}
 
 	before := time.Now()
-	result := s.profileHandler.ProcessResults(ctx, request, profileRunResults)
-	metrics.RecordSchedulerPluginProcessingLatency(framework.ProcessProfilesResultsType, s.profileHandler.Name(), time.Since(before))
+	result, err := s.profileHandler.ProcessResults(ctx, cycleState, request, profileRunResults)
+	metrics.RecordSchedulerPluginProcessingLatency(framework.ProcessProfilesResultsType, s.profileHandler.Type(), time.Since(before))
 
-	return result, nil
+	return result, err
 }
diff --git a/pkg/epp/scheduling/scheduler_config.go b/pkg/epp/scheduling/scheduler_config.go
index 4e83aa8..41524f6 100644
--- a/pkg/epp/scheduling/scheduler_config.go
+++ b/pkg/epp/scheduling/scheduler_config.go
@@ -17,13 +17,6 @@ limitations under the License.
 package scheduling
 
 import (
-	"errors"
-	"fmt"
-
-	"github.com/go-logr/logr"
-
-	"sigs.k8s.io/gateway-api-inference-extension/api/config/v1alpha1"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 )
 
@@ -40,48 +33,3 @@ type SchedulerConfig struct {
 	profileHandler framework.ProfileHandler
 	profiles       map[string]*framework.SchedulerProfile
 }
-
-func LoadSchedulerConfig(configProfiles []v1alpha1.SchedulingProfile, references map[string]plugins.Plugin,
-	log logr.Logger) (*SchedulerConfig, error) {
-
-	var profiles = map[string]*framework.SchedulerProfile{}
-
-	for _, configProfile := range configProfiles {
-		profile := framework.SchedulerProfile{}
-
-		for _, plugin := range configProfile.Plugins {
-			var err error
-			thePlugin := references[plugin.PluginRef]
-			if theScorer, ok := thePlugin.(framework.Scorer); ok {
-				if plugin.Weight == nil {
-					err = fmt.Errorf("scorer %s is missing a weight", plugin.PluginRef)
-					log.Error(err, "failed to instantiate scheduler profile")
-					return nil, err
-				}
-				thePlugin = framework.NewWeightedScorer(theScorer, *plugin.Weight)
-			}
-			err = profile.AddPlugins(thePlugin)
-			if err != nil {
-				return nil, err
-			}
-		}
-		profiles[configProfile.Name] = &profile
-	}
-
-	var profileHandler framework.ProfileHandler
-	var profileHandlerName string
-
-	for pluginName, thePlugin := range references {
-		if theProfileHandler, ok := thePlugin.(framework.ProfileHandler); ok {
-			if profileHandler != nil {
-				return nil, fmt.Errorf("only one profile handler is allowed. Both %s and %s are profile handlers", profileHandlerName, pluginName)
-			}
-			profileHandler = theProfileHandler
-			profileHandlerName = pluginName
-		}
-	}
-	if profileHandler != nil {
-		return NewSchedulerConfig(profileHandler, profiles), nil
-	}
-	return nil, errors.New("no profile handler was specified")
-}
diff --git a/site-src/concepts/design-principles.md b/site-src/concepts/design-principles.md
new file mode 100644
index 0000000..76c3b77
--- /dev/null
+++ b/site-src/concepts/design-principles.md
@@ -0,0 +1,47 @@
+# Design Principles
+
+These principles guide our efforts to build flexible [Gateway API] extensions
+that empower the development of high-performance [AI Inference] routing
+technologies—balancing rapid delivery with long-term growth.
+
+!!! note "Inference Gateways"
+
+    For simplicity, we'll refer to Gateway API Gateways which are
+    composed together with AI Inference extensions as "Inference Gateways"
+    throughout this document.
+
+[Gateway API]:https://github.com/kubernetes-sigs/gateway-api
+[AI Inference]:https://www.arm.com/glossary/ai-inference
+
+
+## Prioritize stability of the core interfaces
+
+The most critical part of this project is the interfaces between components. To encourage both controller and extension developers to integrate with this project, we need to prioritize the stability of these interfaces.
+Although we can extend these interfaces in the future, it’s critical the core is stable as soon as possible.
+
+When describing "core interfaces", we are referring to both of the following:
+
+### 1. Gateway -> Endpoint Picker
+At a high level, this defines how a Gateway provides information to an Endpoint Picker, and how the Endpoint Picker selects endpoint(s) that the Gateway should route to.
+
+### 2. Endpoint Picker -> Model Server Framework
+This defines what an Endpoint Picker should expect from a compatible Model Server Framework with a focus on health checks and metrics.
+
+## Our presets are finely tuned
+
+We provide APIs and reference implementations for the most common inference requirements. Our defaults for those APIs and implementations—shaped by extensive experience with leading model serving platforms and APIs—are designed to provide the majority of Inference Gateway users with a great default experience without the need for extensive configuration or customization. If you take all of our default extensions and attach them to a compatible `Gateway`, it just "works out of the box".
+
+## Encourage innovation via extensibility
+
+This project is largely based on the idea that extensibility will enable innovation. With that in mind, we should make it as easy as possible for AI researchers to experiment with custom scheduling and routing logic. They should not need to know how to build a Kubernetes controller, or replicate a full networking stack. Instead, all the information needed to make a routing decision should be provided in an accessible format, with clear guidelines and examples of how to customize routing logic.
+
+## Objectives over instructions
+
+The pace of innovation in this ecosystem has been rapid. Focusing too heavily on the specifics of current techniques could result in the API becoming outdated quickly. Instead of making the API too descriptive about _how_ an objective should be achieved, this API should focus on the objectives that a Gateway and/or Endpoint Picker should strive to attain. Overly specific instructions or configuration can start as implementation specific APIs and grow into standards as the concepts become more stable and widespread.
+
+## Composable components and reducing reinvention
+While it may be tempting to develop an entirely new AI-focused Gateway, many essential routing capabilities are already well established by Kubernetes. Our focus is on creating a layer of composable components that can be assembled together with other Kubernetes components. This approach empowers engineers to use our solution as a building block—combining established technologies like Gateway API with our extensible model to build higher level solutions. Should you encounter a limitation, consider how existing tooling may be extended or improved first.
+
+## Additions to the API should be carefully prioritized
+
+Every addition to the API should take the principles described above into account. Given that the goal of the API is to encourage a highly extensible ecosystem, each additional feature in the API is raising the barrier for entry to any new controller or extension. Our top priority should be to focus on concepts that we expect to be broadly implementable and useful. The extensible nature of this API will allow each individual implementation to experiment with new features via custom flags or APIs before they become part of the core API surface.
diff --git a/site-src/guides/index.md b/site-src/guides/index.md
index 571f541..04b781b 100644
--- a/site-src/guides/index.md
+++ b/site-src/guides/index.md
@@ -74,8 +74,7 @@ This quickstart guide is intended for engineers familiar with k8s and model serv
 === "Latest Release"
 
       ```bash
-      VERSION=v0.3.0
-      kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/download/$VERSION/manifests.yaml
+      kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/latest/download/manifests.yaml
       ```
 
 === "Dev Version"
diff --git a/site-src/guides/metrics.md b/site-src/guides/metrics.md
index 9f49fe7..b3ebca8 100644
--- a/site-src/guides/metrics.md
+++ b/site-src/guides/metrics.md
@@ -92,9 +92,9 @@ subjects:
   name: inference-gateway-sa-metrics-reader
   namespace: default
 roleRef:
+  apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: inference-gateway-metrics-reader
-  apiGroup: rbac.authorization.k8s.io
 ---
 apiVersion: v1
 kind: Secret
diff --git a/tools/dynamic-lora-sidecar/Dockerfile b/tools/dynamic-lora-sidecar/Dockerfile
index 4faf360..14703d1 100644
--- a/tools/dynamic-lora-sidecar/Dockerfile
+++ b/tools/dynamic-lora-sidecar/Dockerfile
@@ -1,4 +1,4 @@
-FROM python:3.9-slim-buster AS test
+FROM python:3.10-slim-buster AS test
 
 WORKDIR /dynamic-lora-reconciler-test
 COPY requirements.txt .
diff --git a/tools/dynamic-lora-sidecar/Makefile b/tools/dynamic-lora-sidecar/Makefile
new file mode 100644
index 0000000..d498750
--- /dev/null
+++ b/tools/dynamic-lora-sidecar/Makefile
@@ -0,0 +1,33 @@
+# Makefile for dynamic-lora-sidecar
+
+PYTHON_VERSION := 3.10
+VENV_DIR := venv
+PYTHON := $(VENV_DIR)/bin/python
+PIP := $(VENV_DIR)/bin/pip
+
+.PHONY: help venv install test clean
+
+help: ## Show available targets
+	@echo "Available targets:"
+	@echo "  venv     - Create virtual environment"
+	@echo "  install  - Install dependencies"
+	@echo "  test     - Run unit tests"
+	@echo "  clean    - Clean up virtual environment"
+
+venv: $(VENV_DIR)/bin/activate ## Create virtual environment
+
+$(VENV_DIR)/bin/activate:
+	python$(PYTHON_VERSION) -m venv $(VENV_DIR)
+
+install: venv ## Install dependencies
+	$(PIP) install --upgrade pip
+	$(PIP) install -r requirements.txt
+
+test: install ## Run unit tests
+	$(PYTHON) -m unittest discover -v -s sidecar
+
+clean: ## Clean up virtual environment
+	rm -rf $(VENV_DIR)
+	rm -rf .pytest_cache
+	find . -name "*.pyc" -delete
+	find . -name "__pycache__" -type d -exec rm -rf {} +
diff --git a/tools/dynamic-lora-sidecar/README.md b/tools/dynamic-lora-sidecar/README.md
index 2904d1e..f50c783 100644
--- a/tools/dynamic-lora-sidecar/README.md
+++ b/tools/dynamic-lora-sidecar/README.md
@@ -1,6 +1,6 @@
 # Dynamic LORA Adapter Sidecar for vLLM
 
-This is a sidecar-based tool to help rolling out new LoRA adapters to a set of running vLLM model servers. The user deploys the sidecar with a vLLM server, and using a ConfigMap, the user can express their intent as to which LoRA adapters they want to have the running vLLM servers to be configure with. The sidecar watches the ConfigMap and sends load/unload requests to the vLLM container to actuate on the user intent. 
+This is a sidecar-based tool to help rolling out new LoRA adapters to a set of running vLLM model servers. The user deploys the sidecar with a vLLM server, and using a ConfigMap, the user can express their intent as to which LoRA adapters they want to have the running vLLM servers to be configure with. The sidecar watches the ConfigMap and sends load/unload requests to the vLLM container to actuate on the user intent.
 
 ## Overview
 
@@ -48,6 +48,17 @@ The sidecar uses the vLLM server's API to load or unload adapters based on the c
    ```
    Do not use subPath, since configmap updates are not reflected in the file
 
+## Development
+
+For local development and testing, use the provided Makefile:
+
+```bash
+make venv      # Create Python 3.10 virtual environment
+make install   # Install dependencies
+make test      # Run unit tests
+make clean     # Clean up
+```
+
 ## Command Line Arguments
 
 The sidecar supports the following command-line arguments:
@@ -59,7 +70,7 @@ The sidecar supports the following command-line arguments:
 - `--config-validation`: Enable config validation (default: True)
 
 ## Configuration Fields
-- `vLLMLoRAConfig`[**required**]  base key 
+- `vLLMLoRAConfig`[**required**]  base key
 - `host` [*optional*] Model server's host. defaults to localhost
 - `port` [*optional*] Model server's port. defaults to 8000
 - `name` [*optional*] Name of this config



Test patch:
diff --git a/conformance/testing-epp/plugins/filter/filter_test.go b/conformance/testing-epp/plugins/filter/filter_test.go
index 1b66489..2c00821 100644
--- a/conformance/testing-epp/plugins/filter/filter_test.go
+++ b/conformance/testing-epp/plugins/filter/filter_test.go
@@ -117,7 +117,7 @@ func TestFilter(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			got := test.filter.Filter(context.Background(), test.req, types.NewCycleState(), test.input)
+			got := test.filter.Filter(context.Background(), types.NewCycleState(), test.req, test.input)
 
 			if diff := cmp.Diff(test.output, got); diff != "" {
 				t.Errorf("Unexpected output (-want +got): %v", diff)
diff --git a/conformance/testing-epp/plugins/filter/request_header_based_filter.go b/conformance/testing-epp/plugins/filter/request_header_based_filter.go
index 76cd0c1..ee19111 100644
--- a/conformance/testing-epp/plugins/filter/request_header_based_filter.go
+++ b/conformance/testing-epp/plugins/filter/request_header_based_filter.go
@@ -44,13 +44,18 @@ func NewHeaderBasedTestingFilter() *HeaderBasedTestingFilter {
 // HeaderBasedTestingFilter filters Pods based on an address specified in the "test-epp-endpoint-selection" request header.
 type HeaderBasedTestingFilter struct{}
 
-// Name returns the name of the filter.
-func (f *HeaderBasedTestingFilter) Name() string {
+// Type returns the type of the filter.
+func (f *HeaderBasedTestingFilter) Type() string {
 	return "header-based-testing"
 }
 
+// Name returns the type of the filter.
+func (f *HeaderBasedTestingFilter) Name() string {
+	return "header-based-testing-filter"
+}
+
 // Filter selects pods that match the IP addresses specified in the request header.
-func (f *HeaderBasedTestingFilter) Filter(_ context.Context, request *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *HeaderBasedTestingFilter) Filter(_ context.Context, _ *types.CycleState, request *types.LLMRequest, pods []types.Pod) []types.Pod {
 	headerValue, ok := request.Headers[headerTestEppEndPointSelectionKey]
 	if !ok || headerValue == "" {
 		return []types.Pod{}
diff --git a/conformance/testing-epp/scheduler.go b/conformance/testing-epp/scheduler.go
index 26028ad..94f9ee5 100644
--- a/conformance/testing-epp/scheduler.go
+++ b/conformance/testing-epp/scheduler.go
@@ -27,11 +27,11 @@ import (
 // NewReqHeaderBasedScheduler creates a scheduler for conformance tests that selects
 // an endpoint based on the "test-epp-endpoint-selection" request header. If the
 // header is missing or the specified endpoint doesn't exist, no endpoint is returned.
-func NewReqHeaderBasedScheduler(datastore scheduling.Datastore) *scheduling.Scheduler {
+func NewReqHeaderBasedScheduler() *scheduling.Scheduler {
 	predicatableSchedulerProfile := framework.NewSchedulerProfile().
 		WithFilters(filter.NewHeaderBasedTestingFilter()).
 		WithPicker(picker.NewMaxScorePicker())
 
-	return scheduling.NewSchedulerWithConfig(datastore, scheduling.NewSchedulerConfig(
+	return scheduling.NewSchedulerWithConfig(scheduling.NewSchedulerConfig(
 		profile.NewSingleProfileHandler(), map[string]*framework.SchedulerProfile{"req-header-based-profile": predicatableSchedulerProfile}))
 }
diff --git a/conformance/testing-epp/sheduler_test.go b/conformance/testing-epp/sheduler_test.go
index 0fb24cf..4901e03 100644
--- a/conformance/testing-epp/sheduler_test.go
+++ b/conformance/testing-epp/sheduler_test.go
@@ -31,13 +31,13 @@ import (
 func TestSchedule(t *testing.T) {
 	tests := []struct {
 		name    string
-		input   []*backendmetrics.FakePodMetrics
+		input   []backendmetrics.PodMetrics
 		req     *types.LLMRequest
 		wantRes *types.SchedulingResult
 		err     bool
 	}{
 		{
-			name: "no pods in datastore and req header is set",
+			name: "no candidate pods and req header is set",
 			req: &types.LLMRequest{
 				Headers:   map[string]string{"test-epp-endpoint-selection": "random-endpoint"},
 				RequestId: uuid.NewString(),
@@ -47,8 +47,8 @@ func TestSchedule(t *testing.T) {
 		},
 		{
 			name: "req header not set",
-			input: []*backendmetrics.FakePodMetrics{
-				{Pod: &backend.Pod{Address: "random-endpoint"}},
+			input: []backendmetrics.PodMetrics{
+				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "random-endpoint"}},
 			},
 			req: &types.LLMRequest{
 				Headers:   map[string]string{}, // Deliberately set an empty header.
@@ -58,9 +58,9 @@ func TestSchedule(t *testing.T) {
 			err:     true,
 		},
 		{
-			name: "no pods address in datastore matches req header address",
-			input: []*backendmetrics.FakePodMetrics{
-				{Pod: &backend.Pod{Address: "nonmatched-endpoint"}},
+			name: "no pods address from the candidate pods matches req header address",
+			input: []backendmetrics.PodMetrics{
+				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "nonmatched-endpoint"}},
 			},
 			req: &types.LLMRequest{
 				Headers:   map[string]string{"test-epp-endpoint-selection": "matched-endpoint"},
@@ -70,10 +70,10 @@ func TestSchedule(t *testing.T) {
 			err:     true,
 		},
 		{
-			name: "one pod address in datastore matches req header address",
-			input: []*backendmetrics.FakePodMetrics{
-				{Pod: &backend.Pod{Address: "nonmatched-endpoint"}},
-				{Pod: &backend.Pod{Address: "matched-endpoint"}},
+			name: "one pod address from the candidate pods matches req header address",
+			input: []backendmetrics.PodMetrics{
+				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "nonmatched-endpoint"}},
+				&backendmetrics.FakePodMetrics{Pod: &backend.Pod{Address: "matched-endpoint"}},
 			},
 			req: &types.LLMRequest{
 				Headers:   map[string]string{"test-epp-endpoint-selection": "matched-endpoint"},
@@ -99,8 +99,8 @@ func TestSchedule(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			scheduler := NewReqHeaderBasedScheduler(&fakeDataStore{pods: test.input})
-			got, err := scheduler.Schedule(context.Background(), test.req)
+			scheduler := NewReqHeaderBasedScheduler()
+			got, err := scheduler.Schedule(context.Background(), test.req, types.ToSchedulerPodMetrics(test.input))
 			if test.err != (err != nil) {
 				t.Errorf("Unexpected error, got %v, want %v", err, test.err)
 			}
@@ -111,15 +111,3 @@ func TestSchedule(t *testing.T) {
 		})
 	}
 }
-
-type fakeDataStore struct {
-	pods []*backendmetrics.FakePodMetrics
-}
-
-func (fds *fakeDataStore) PodGetAll() []backendmetrics.PodMetrics {
-	pm := make([]backendmetrics.PodMetrics, 0, len(fds.pods))
-	for _, pod := range fds.pods {
-		pm = append(pm, pod)
-	}
-	return pm
-}
diff --git a/conformance/tests/basic/epp_unavailable_fail_open.go b/conformance/tests/basic/epp_unavailable_fail_open.go
new file mode 100644
index 0000000..2f68606
--- /dev/null
+++ b/conformance/tests/basic/epp_unavailable_fail_open.go
@@ -0,0 +1,117 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package basic
+
+import (
+	"net/http"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"k8s.io/apimachinery/pkg/types"
+	"sigs.k8s.io/gateway-api/conformance/utils/suite"
+	"sigs.k8s.io/gateway-api/pkg/features"
+
+	"sigs.k8s.io/gateway-api-inference-extension/conformance/tests"
+	k8sutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
+	trafficutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/traffic"
+)
+
+func init() {
+	tests.ConformanceTests = append(tests.ConformanceTests, EppUnAvailableFailOpen)
+}
+
+var EppUnAvailableFailOpen = suite.ConformanceTest{
+	ShortName:   "EppUnAvailableFailOpen",
+	Description: "Inference gateway should send traffic to backends even when the EPP is unavailable (fail-open)",
+	Manifests:   []string{"tests/basic/epp_unavailable_fail_open.yaml"},
+	Features: []features.FeatureName{
+		features.FeatureName("SupportInferencePool"),
+		features.SupportGateway,
+	},
+	Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
+		const (
+			appBackendNamespace    = "gateway-conformance-app-backend"
+			infraNamespace         = "gateway-conformance-infra"
+			hostname               = "secondary.example.com"
+			path                   = "/failopen-pool-test"
+			expectedPodReplicas    = 3
+			eppSelectionHeaderName = "test-epp-endpoint-selection"
+			appPodBackendPrefix    = "secondary-inference-model-server"
+			requestBody            = `{
+                "model": "conformance-fake-model",
+                "prompt": "Write as if you were a critic: San Francisco"
+            }`
+		)
+
+		httpRouteNN := types.NamespacedName{Name: "httproute-for-failopen-pool-gw", Namespace: appBackendNamespace}
+		gatewayNN := types.NamespacedName{Name: "conformance-secondary-gateway", Namespace: infraNamespace}
+		poolNN := types.NamespacedName{Name: "secondary-inference-pool", Namespace: appBackendNamespace}
+		eppDeploymentNN := types.NamespacedName{Name: "secondary-app-endpoint-picker", Namespace: appBackendNamespace}
+		backendPodLabels := map[string]string{"app": "secondary-inference-model-server"}
+
+		k8sutils.HTTPRouteMustBeAcceptedAndResolved(t, s.Client, s.TimeoutConfig, httpRouteNN, gatewayNN)
+		k8sutils.InferencePoolMustBeAcceptedByParent(t, s.Client, poolNN)
+		gwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, gatewayNN)
+
+		pods, err := k8sutils.GetPodsWithLabel(t, s.Client, appBackendNamespace, backendPodLabels)
+		require.NoError(t, err, "Failed to get backend pods")
+		require.Len(t, pods, expectedPodReplicas, "Expected to find %d backend pod, but found %d.", expectedPodReplicas, len(pods))
+
+		targetPodIP := pods[0].Status.PodIP
+		t.Run("Phase 1: Verify baseline connectivity with EPP available", func(t *testing.T) {
+			t.Log("Sending request to ensure the Gateway and EPP are working correctly...")
+			trafficutils.MakeRequestWithRequestParamAndExpectSuccess(
+				t,
+				s.RoundTripper,
+				s.TimeoutConfig,
+				gwAddr,
+				trafficutils.Request{
+					Host:      hostname,
+					Path:      path,
+					Headers:   map[string]string{eppSelectionHeaderName: targetPodIP},
+					Method:    http.MethodPost,
+					Body:      requestBody,
+					Backend:   pods[0].Name, // Make sure the request is from the targetPod when the EPP is alive.
+					Namespace: appBackendNamespace,
+				},
+			)
+		})
+
+		t.Run("Phase 2: Verify fail-open behavior after EPP becomes unavailable", func(t *testing.T) {
+			t.Log("Simulating an EPP failure by deleting its deployment...")
+			deleteErr := k8sutils.DeleteDeployment(t, s.Client, s.TimeoutConfig, eppDeploymentNN)
+			require.NoError(t, deleteErr, "Failed to delete the EPP deployment")
+
+			t.Log("Sending request again, expecting success to verify fail-open...")
+			trafficutils.MakeRequestWithRequestParamAndExpectSuccess(
+				t,
+				s.RoundTripper,
+				s.TimeoutConfig,
+				gwAddr,
+				trafficutils.Request{
+					Host:      hostname,
+					Path:      path,
+					Headers:   map[string]string{eppSelectionHeaderName: targetPodIP},
+					Method:    http.MethodPost,
+					Body:      requestBody,
+					Backend:   appPodBackendPrefix, // Only checks the prefix since the EPP is not alive and the response can return from any Pod.
+					Namespace: appBackendNamespace,
+				},
+			)
+		})
+	},
+}
diff --git a/conformance/tests/basic/epp_unavailable_fail_open.yaml b/conformance/tests/basic/epp_unavailable_fail_open.yaml
new file mode 100644
index 0000000..cd681cb
--- /dev/null
+++ b/conformance/tests/basic/epp_unavailable_fail_open.yaml
@@ -0,0 +1,36 @@
+# --- InferenceModel Definition ---
+# TODO: remove inferenceModel dependency https://github.com/kubernetes-sigs/gateway-api-inference-extension/issues/1002
+apiVersion: inference.networking.x-k8s.io/v1alpha2
+kind: InferenceModel
+metadata:
+  name: conformance-fake-model-server
+  namespace: gateway-conformance-app-backend
+spec:
+  modelName: conformance-fake-model
+  criticality: Critical # Mark it as critical to bypass the saturation check since the model server is fake and don't have such metrics. 
+  poolRef:
+    name: secondary-inference-pool
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: httproute-for-failopen-pool-gw
+  namespace: gateway-conformance-app-backend
+spec:
+  parentRefs:
+  - group: gateway.networking.k8s.io
+    kind: Gateway
+    name: conformance-secondary-gateway
+    namespace: gateway-conformance-infra
+    sectionName: http
+  hostnames:
+  - "secondary.example.com"
+  rules:
+  - backendRefs:
+    - group: inference.networking.x-k8s.io
+      kind: InferencePool
+      name: secondary-inference-pool # Use secondary-inferencePool because it has failureMode set to failOpen
+    matches:
+    - path:
+        type: PathPrefix
+        value: /failopen-pool-test
diff --git a/conformance/tests/basic/gateway_following_epp_routing.go b/conformance/tests/basic/gateway_following_epp_routing.go
new file mode 100644
index 0000000..9877999
--- /dev/null
+++ b/conformance/tests/basic/gateway_following_epp_routing.go
@@ -0,0 +1,194 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package basic
+
+import (
+	"fmt"
+	"net/http"
+	"slices"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"golang.org/x/sync/errgroup"
+	"k8s.io/apimachinery/pkg/types"
+	"sigs.k8s.io/gateway-api/conformance/utils/suite"
+	"sigs.k8s.io/gateway-api/pkg/features"
+
+	"sigs.k8s.io/gateway-api-inference-extension/conformance/tests"
+	k8sutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
+	"sigs.k8s.io/gateway-api-inference-extension/conformance/utils/traffic"
+	trafficutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/traffic"
+	gwhttp "sigs.k8s.io/gateway-api/conformance/utils/http"
+)
+
+func init() {
+	// Register the GatewayFollowingEPPRouting test case with the conformance suite.
+	// This ensures it will be discovered and run by the test runner.
+	tests.ConformanceTests = append(tests.ConformanceTests, GatewayFollowingEPPRouting)
+}
+
+// GatewayFollowingEPPRouting defines the test case for verifying gateway should send traffic to an endpoint in the list returned by EPP.
+var GatewayFollowingEPPRouting = suite.ConformanceTest{
+	ShortName:   "GatewayFollowingEPPRouting",
+	Description: "Inference gateway should send traffic to an endpoint in the list returned by EPP",
+	Manifests:   []string{"tests/basic/gateway_following_epp_routing.yaml"},
+	Features: []features.FeatureName{
+		features.FeatureName("SupportInferencePool"),
+		features.SupportGateway,
+	},
+	Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
+		const (
+			appBackendNamespace = "gateway-conformance-app-backend"
+			infraNamespace      = "gateway-conformance-infra"
+			hostname            = "primary.example.com"
+			path                = "/primary-gateway-test"
+			expectedPodReplicas = 3
+			// eppSelectionHeaderName is the custom header used by the testing-EPP service
+			// to determine which endpoint to select.
+			eppSelectionHeaderName = "test-epp-endpoint-selection"
+			appPodBackendPrefix    = "primary-inference-model-server"
+		)
+
+		httpRouteNN := types.NamespacedName{Name: "httproute-for-primary-gw", Namespace: appBackendNamespace}
+		gatewayNN := types.NamespacedName{Name: "conformance-primary-gateway", Namespace: infraNamespace}
+		poolNN := types.NamespacedName{Name: "primary-inference-pool", Namespace: appBackendNamespace}
+		backendPodLabels := map[string]string{"app": "primary-inference-model-server"}
+
+		t.Log("Verifying HTTPRoute and InferencePool are accepted and the Gateway has an address.")
+		k8sutils.HTTPRouteMustBeAcceptedAndResolved(t, s.Client, s.TimeoutConfig, httpRouteNN, gatewayNN)
+		k8sutils.InferencePoolMustBeAcceptedByParent(t, s.Client, poolNN)
+		gwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, gatewayNN)
+
+		t.Logf("Fetching backend pods with labels: %v", backendPodLabels)
+		pods, err := k8sutils.GetPodsWithLabel(t, s.Client, appBackendNamespace, backendPodLabels)
+		require.NoError(t, err, "Failed to get backend pods")
+		require.Len(t, pods, expectedPodReplicas, "Expected to find %d backend pods, but found %d.", expectedPodReplicas, len(pods))
+
+		podIPs := make([]string, len(pods))
+		podNames := make([]string, len(pods))
+		for i, pod := range pods {
+			podIPs[i] = pod.Status.PodIP
+			podNames[i] = pod.Name
+		}
+
+		requestBody := `{
+            "model": "conformance-fake-model",
+            "prompt": "Write as if you were a critic: San Francisco"
+        }`
+
+		for i := 0; i < len(pods); i++ {
+			// Send an initial request targeting a single pod and wait for it to be successful to ensure the Gateway and EPP
+			// are functioning correctly before running the main test cases.
+			trafficutils.MakeRequestWithRequestParamAndExpectSuccess(
+				t,
+				s.RoundTripper,
+				s.TimeoutConfig,
+				gwAddr,
+				trafficutils.Request{
+					Host:      hostname,
+					Path:      path,
+					Headers:   map[string]string{eppSelectionHeaderName: podIPs[i]},
+					Method:    http.MethodPost,
+					Body:      requestBody,
+					Backend:   podNames[i],
+					Namespace: appBackendNamespace,
+				},
+			)
+		}
+
+		testCases := []struct {
+			name                                  string
+			podIPsToBeReturnedByEPP               []string
+			expectAllRequestsRoutedWithinPodNames []string
+		}{
+			{
+				name:                                  "should route traffic to a single designated pod",
+				podIPsToBeReturnedByEPP:               []string{podIPs[2]},
+				expectAllRequestsRoutedWithinPodNames: []string{podNames[2]},
+			},
+			{
+				name:                                  "should route traffic to two designated pods",
+				podIPsToBeReturnedByEPP:               []string{podIPs[0], podIPs[1]},
+				expectAllRequestsRoutedWithinPodNames: []string{podNames[0], podNames[1]},
+			},
+			{
+				name:                                  "should route traffic to all available pods",
+				podIPsToBeReturnedByEPP:               []string{podIPs[0], podIPs[1], podIPs[2]},
+				expectAllRequestsRoutedWithinPodNames: []string{podNames[0], podNames[1], podNames[2]},
+			},
+		}
+
+		for _, tc := range testCases {
+			t.Run(tc.name, func(t *testing.T) {
+				eppHeaderValue := strings.Join(tc.podIPsToBeReturnedByEPP, ",")
+				headers := map[string]string{eppSelectionHeaderName: eppHeaderValue}
+
+				t.Logf("Sending request to %s with EPP header '%s: %s'", gwAddr, eppSelectionHeaderName, eppHeaderValue)
+				t.Logf("Expecting traffic to be routed to pod: %v", tc.expectAllRequestsRoutedWithinPodNames)
+
+				assertTrafficOnlyReachesToExpectedPods(t, s, gwAddr, gwhttp.ExpectedResponse{
+					Request: gwhttp.Request{
+						Host:    hostname,
+						Path:    path,
+						Method:  http.MethodPost,
+						Headers: headers,
+					},
+					Response: gwhttp.Response{
+						StatusCode: http.StatusOK,
+					},
+					Backend:   appPodBackendPrefix,
+					Namespace: appBackendNamespace,
+				}, requestBody, tc.expectAllRequestsRoutedWithinPodNames)
+			})
+		}
+	},
+}
+
+func assertTrafficOnlyReachesToExpectedPods(t *testing.T, suite *suite.ConformanceTestSuite, gwAddr string, expected gwhttp.ExpectedResponse, requestBody string, expectedPodNames []string) {
+	t.Helper()
+	const (
+		concurrentRequests = 10
+		totalRequests      = 100
+	)
+	var (
+		roundTripper = suite.RoundTripper
+		g            errgroup.Group
+		req          = gwhttp.MakeRequest(t, &expected, gwAddr, "HTTP", "http")
+	)
+	g.SetLimit(concurrentRequests)
+	for i := 0; i < totalRequests; i++ {
+		g.Go(func() error {
+			cReq, cRes, err := traffic.MakeCallRoundTripper(t, roundTripper, &traffic.RequestWithBody{Request: req, Body: strings.NewReader(requestBody)})
+			if err != nil {
+				return fmt.Errorf("failed to roundtrip request: %w", err)
+			}
+			if err := gwhttp.CompareRequest(t, &req, cReq, cRes, expected); err != nil {
+				return fmt.Errorf("response expectation failed for request: %w", err)
+			}
+
+			if slices.Contains(expectedPodNames, cReq.Pod) {
+				return nil
+			}
+			return fmt.Errorf("request was handled by an unexpected pod %q", cReq.Pod)
+		})
+	}
+	if err := g.Wait(); err != nil {
+		t.Fatalf("Not all the requests are sent to the expectedPods successfully, err: %v", err)
+	}
+	t.Logf("Traffic successfully reached only to expected pods: %v", expectedPodNames)
+}
diff --git a/conformance/tests/basic/gateway_following_epp_routing.yaml b/conformance/tests/basic/gateway_following_epp_routing.yaml
new file mode 100644
index 0000000..d290b75
--- /dev/null
+++ b/conformance/tests/basic/gateway_following_epp_routing.yaml
@@ -0,0 +1,38 @@
+# --- InferenceModel Definition ---
+# Service for the infra-backend-deployment.
+apiVersion: inference.networking.x-k8s.io/v1alpha2
+kind: InferenceModel
+metadata:
+  name: conformance-fake-model-server
+  namespace: gateway-conformance-app-backend
+spec:
+  modelName: conformance-fake-model
+  criticality: Critical # Mark it as critical to bypass the saturation check since the model server is fake and don't have such metrics. 
+  poolRef:
+    name: primary-inference-pool
+---
+# --- HTTPRoute for Primary Gateway (conformance-gateway) ---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: httproute-for-primary-gw
+  namespace: gateway-conformance-app-backend
+spec:
+  parentRefs:
+  - group: gateway.networking.k8s.io
+    kind: Gateway
+    name: conformance-primary-gateway
+    namespace: gateway-conformance-infra
+    sectionName: http
+  hostnames:
+  - "primary.example.com"
+  rules:
+  - backendRefs:
+    - group: inference.networking.x-k8s.io
+      kind: InferencePool
+      name: primary-inference-pool
+    matches:
+    - path:
+        type: PathPrefix
+        value: /primary-gateway-test
+  
\ No newline at end of file
diff --git a/conformance/tests/basic/httproute_multiple_gateways_different_pools.go b/conformance/tests/basic/httproute_multiple_gateways_different_pools.go
new file mode 100644
index 0000000..251f55c
--- /dev/null
+++ b/conformance/tests/basic/httproute_multiple_gateways_different_pools.go
@@ -0,0 +1,118 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package basic
+
+import (
+	"net/http"
+	"testing"
+
+	"k8s.io/apimachinery/pkg/types"
+	"sigs.k8s.io/gateway-api/conformance/utils/suite"
+
+	"sigs.k8s.io/gateway-api-inference-extension/conformance/tests"
+	k8sutils "sigs.k8s.io/gateway-api-inference-extension/conformance/utils/kubernetes"
+	"sigs.k8s.io/gateway-api-inference-extension/conformance/utils/traffic"
+)
+
+func init() {
+	tests.ConformanceTests = append(tests.ConformanceTests, HTTPRouteMultipleGatewaysDifferentPools)
+}
+
+var HTTPRouteMultipleGatewaysDifferentPools = suite.ConformanceTest{
+	ShortName:   "HTTPRouteMultipleGatewaysDifferentPools",
+	Description: "Validates two HTTPRoutes on different Gateways successfully referencing different InferencePools and routes traffic accordingly.",
+	Manifests:   []string{"tests/basic/httproute_multiple_gateways_different_pools.yaml"},
+	Test: func(t *testing.T, s *suite.ConformanceTestSuite) {
+		const (
+			appBackendNamespace = "gateway-conformance-app-backend"
+			infraNamespace      = "gateway-conformance-infra"
+
+			primaryGatewayName    = "conformance-primary-gateway"
+			routeForPrimaryGWName = "route-for-primary-gateway"
+			primaryPoolName       = "primary-inference-pool"
+			primaryBackendPodName = "primary-inference-model-server"
+			primaryRoutePath      = "/test-primary-gateway"
+			primaryRouteHostname  = "primary.example.com"
+
+			secondaryGatewayName    = "conformance-secondary-gateway"
+			routeForSecondaryGWName = "route-for-secondary-gateway"
+			secondaryPoolName       = "secondary-inference-pool"
+			secondaryBackendPodName = "secondary-inference-model-server"
+			secondaryRoutePath      = "/test-secondary-gateway"
+			secondaryRouteHostname  = "secondary.example.com"
+		)
+
+		routeForPrimaryGWNN := types.NamespacedName{Name: routeForPrimaryGWName, Namespace: appBackendNamespace}
+		routeForSecondaryGWNN := types.NamespacedName{Name: routeForSecondaryGWName, Namespace: appBackendNamespace}
+		primaryPoolNN := types.NamespacedName{Name: primaryPoolName, Namespace: appBackendNamespace}
+		secondaryPoolNN := types.NamespacedName{Name: secondaryPoolName, Namespace: appBackendNamespace}
+		primaryGatewayNN := types.NamespacedName{Name: primaryGatewayName, Namespace: infraNamespace}
+		secondaryGatewayNN := types.NamespacedName{Name: secondaryGatewayName, Namespace: infraNamespace}
+
+		t.Run("Primary HTTPRoute, InferencePool, and Gateway path: verify status and traffic", func(t *testing.T) {
+			k8sutils.HTTPRouteAndInferencePoolMustBeAcceptedAndRouteAccepted(
+				t,
+				s.Client,
+				routeForPrimaryGWNN,
+				primaryGatewayNN,
+				primaryPoolNN,
+			)
+
+			primaryGwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, primaryGatewayNN)
+
+			traffic.MakeRequestAndExpectEventuallyConsistentResponse(
+				t,
+				s.RoundTripper,
+				s.TimeoutConfig,
+				primaryGwAddr,
+				traffic.Request{
+					Host:               primaryRouteHostname,
+					Path:               primaryRoutePath,
+					ExpectedStatusCode: http.StatusOK,
+					Backend:            primaryBackendPodName,
+					Namespace:          appBackendNamespace,
+				},
+			)
+		})
+
+		t.Run("Secondary HTTPRoute, InferencePool, and Gateway path: verify status and traffic", func(t *testing.T) {
+			k8sutils.HTTPRouteAndInferencePoolMustBeAcceptedAndRouteAccepted(
+				t,
+				s.Client,
+				routeForSecondaryGWNN,
+				secondaryGatewayNN,
+				secondaryPoolNN,
+			)
+
+			secondaryGwAddr := k8sutils.GetGatewayEndpoint(t, s.Client, s.TimeoutConfig, secondaryGatewayNN)
+
+			traffic.MakeRequestAndExpectEventuallyConsistentResponse(
+				t,
+				s.RoundTripper,
+				s.TimeoutConfig,
+				secondaryGwAddr,
+				traffic.Request{
+					Host:               secondaryRouteHostname,
+					Path:               secondaryRoutePath,
+					ExpectedStatusCode: http.StatusOK,
+					Backend:            secondaryBackendPodName,
+					Namespace:          appBackendNamespace,
+				},
+			)
+		})
+	},
+}
diff --git a/conformance/tests/basic/httproute_multiple_gateways_different_pools.yaml b/conformance/tests/basic/httproute_multiple_gateways_different_pools.yaml
new file mode 100644
index 0000000..b8ec2d1
--- /dev/null
+++ b/conformance/tests/basic/httproute_multiple_gateways_different_pools.yaml
@@ -0,0 +1,44 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: route-for-primary-gateway
+  namespace: gateway-conformance-app-backend
+spec:
+  parentRefs:
+  - kind: Gateway
+    name: conformance-primary-gateway
+    namespace: gateway-conformance-infra
+  hostnames:
+  - "primary.example.com"
+  rules:
+  - backendRefs:
+    - group: inference.networking.x-k8s.io
+      kind: InferencePool
+      name: primary-inference-pool
+    matches:
+    - path:
+        type: PathPrefix
+        value: /test-primary-gateway
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: route-for-secondary-gateway
+  namespace: gateway-conformance-app-backend
+spec:
+  parentRefs:
+  - kind: Gateway
+    name: conformance-secondary-gateway
+    namespace: gateway-conformance-infra
+  hostnames:
+  - "secondary.example.com"
+  rules:
+  - backendRefs:
+    - group: inference.networking.x-k8s.io
+      kind: InferencePool
+      name: secondary-inference-pool
+    matches:
+    - path:
+        type: PathPrefix
+        value: /test-secondary-gateway
diff --git a/conformance/tests/basic/inferencepool_httproute_port_validation.go b/conformance/tests/basic/inferencepool_httproute_port_validation.go
index 888b659..45d9ed7 100644
--- a/conformance/tests/basic/inferencepool_httproute_port_validation.go
+++ b/conformance/tests/basic/inferencepool_httproute_port_validation.go
@@ -45,9 +45,9 @@ var InferencePoolHTTPRoutePortValidation = suite.ConformanceTest{
 		const (
 			appBackendNamespace   = "gateway-conformance-app-backend"
 			infraNamespace        = "gateway-conformance-infra"
-			gatewayName           = "conformance-gateway"
-			poolName              = "target-pool-port-validation"
-			backendDeploymentName = "infra-backend-deployment-port-test"
+			gatewayName           = "conformance-primary-gateway"
+			poolName              = "primary-inference-pool"
+			backendDeploymentName = "primary-inference-model-server-deployment"
 		)
 
 		gatewayNN := types.NamespacedName{Name: gatewayName, Namespace: infraNamespace}
diff --git a/conformance/tests/basic/inferencepool_httproute_port_validation.yaml b/conformance/tests/basic/inferencepool_httproute_port_validation.yaml
index d15d317..62e243a 100644
--- a/conformance/tests/basic/inferencepool_httproute_port_validation.yaml
+++ b/conformance/tests/basic/inferencepool_httproute_port_validation.yaml
@@ -1,143 +1,5 @@
-# conformance/tests/basic/inferencepool_httproute_port_validation.yaml
-
-# --- Backend Deployment (reusing standard echoserver) ---
-
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: infra-backend-deployment-port-test
-  namespace: gateway-conformance-app-backend
-  labels:
-    app: infra-backend-port-test
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: infra-backend-port-test
-  template:
-    metadata:
-      labels:
-        app: infra-backend-port-test
-    spec:
-      containers:
-      - name: echoserver
-        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
-        ports:
-        - containerPort: 3000
-        readinessProbe:
-          httpGet:
-            path: /
-            port: 3000
-          initialDelaySeconds: 3
-          periodSeconds: 5
-          failureThreshold: 2
-        env:
-        - name: POD_NAME
-          valueFrom:
-            fieldRef:
-              fieldPath: metadata.name
-        - name: NAMESPACE
-          valueFrom:
-            fieldRef:
-              fieldPath: metadata.namespace
----
-# --- Backend Service ---
-# Service for the infra-backend-deployment-port-test.
-apiVersion: v1
-kind: Service
-metadata:
-  name: infra-backend-svc-port-test
-  namespace: gateway-conformance-app-backend
-spec:
-  selector:
-    app: infra-backend-port-test
-  ports:
-  - name: http
-    port: 3000
-    targetPort: 3000
----
-# --- InferencePool Definition ---
-apiVersion: inference.networking.x-k8s.io/v1alpha2
-kind: InferencePool
-metadata:
-  name: target-pool-port-validation
-  namespace: gateway-conformance-app-backend
-spec:
-  selector:
-    app: "infra-backend-port-test"
-  targetPortNumber: 3000
-  extensionRef:
-    name: target-pool-port-validation-epp
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: target-pool-port-validation-epp
-  namespace: gateway-conformance-app-backend
-spec:
-  selector:
-    app: target-pool-port-validation-epp
-  ports:
-    - protocol: TCP
-      port: 9002
-      targetPort: 9002
-      appProtocol: http2
-  type: ClusterIP
----
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: target-pool-port-validation-epp
-  namespace: gateway-conformance-app-backend
-  labels:
-    app: target-pool-port-validation-epp
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: target-pool-port-validation-epp
-  template:
-    metadata:
-      labels:
-        app: target-pool-port-validation-epp
-    spec:
-      terminationGracePeriodSeconds: 130
-      containers:
-        - name: epp
-          image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:main
-          imagePullPolicy: Always
-          args:
-            - -poolName
-            - "target-pool-port-validation"
-            - "-poolNamespace"
-            - "gateway-conformance-app-backend"
-            - -v
-            - "4"
-            - --zap-encoder
-            - "json"
-            - -grpcPort
-            - "9002"
-            - -grpcHealthPort
-            - "9003"
-          ports:
-            - containerPort: 9002
-            - containerPort: 9003
-            - name: metrics
-              containerPort: 9090
-          livenessProbe:
-            grpc:
-              port: 9003
-              service: inference-extension
-            initialDelaySeconds: 5
-            periodSeconds: 10
-          readinessProbe:
-            grpc:
-              port: 9003
-              service: inference-extension
-            initialDelaySeconds: 5
-            periodSeconds: 10
----
 # --- HTTPRoute Scenario 1: Port Unspecified ---
+---
 apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
@@ -147,7 +9,7 @@ spec:
   parentRefs:
   - group: gateway.networking.k8s.io
     kind: Gateway
-    name: conformance-gateway
+    name: conformance-primary-gateway
     namespace: gateway-conformance-infra
     sectionName: http
   hostnames:
@@ -156,7 +18,7 @@ spec:
   - backendRefs:
     - group: inference.networking.x-k8s.io
       kind: InferencePool
-      name: target-pool-port-validation
+      name: primary-inference-pool
       # Port is intentionally unspecified here
     matches:
     - path:
@@ -173,7 +35,7 @@ spec:
   parentRefs:
   - group: gateway.networking.k8s.io
     kind: Gateway
-    name: conformance-gateway
+    name: conformance-primary-gateway
     namespace: gateway-conformance-infra
     sectionName: http
   hostnames:
@@ -182,7 +44,7 @@ spec:
   - backendRefs:
     - group: inference.networking.x-k8s.io
       kind: InferencePool
-      name: target-pool-port-validation
+      name: primary-inference-pool
       port: 3000 # Port matches InferencePool's targetPortNumber
     matches:
     - path:
@@ -199,7 +61,7 @@ spec:
   parentRefs:
   - group: gateway.networking.k8s.io
     kind: Gateway
-    name: conformance-gateway
+    name: conformance-primary-gateway
     namespace: gateway-conformance-infra
     sectionName: http
   hostnames:
@@ -208,37 +70,10 @@ spec:
   - backendRefs:
     - group: inference.networking.x-k8s.io
       kind: InferencePool
-      name: target-pool-port-validation
+      name: primary-inference-pool
       port: 8888 # Port does NOT match InferencePool's targetPortNumber
     matches:
     - path:
         type: PathPrefix
         value: /test-port-non-matching
 ---
-# --- Conformance EPP Requried Role and RoleBindings ---
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  name: inference-model-reader
-  namespace: gateway-conformance-app-backend
-rules:
-- apiGroups: ["inference.networking.x-k8s.io"]
-  resources: ["inferencemodels", "inferencepools"]
-  verbs: ["get", "list", "watch"]
-- apiGroups: [""]
-  resources: ["pods"]
-  verbs: ["get", "list", "watch"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  name: epp-to-inference-model-reader
-  namespace: gateway-conformance-app-backend
-subjects:
-- kind: ServiceAccount
-  name: default
-  namespace: gateway-conformance-app-backend
-roleRef:
-  kind: Role
-  name: inference-model-reader
-  apiGroup: rbac.authorization.k8s.io
diff --git a/pkg/epp/common/config/configloader_test.go b/pkg/epp/common/config/loader/configloader_test.go
similarity index 55%
rename from pkg/epp/common/config/configloader_test.go
rename to pkg/epp/common/config/loader/configloader_test.go
index 7600ba5..251c3f6 100644
--- a/pkg/epp/common/config/configloader_test.go
+++ b/pkg/epp/common/config/loader/configloader_test.go
@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package config
+package loader
 
 import (
 	"context"
@@ -27,14 +27,19 @@ import (
 	configapi "sigs.k8s.io/gateway-api-inference-extension/api/config/v1alpha1"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/filter"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/multi/prefix"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
+	"sigs.k8s.io/gateway-api-inference-extension/test/utils"
 )
 
 const (
-	testProfileHandlerName = "test-profile-handler"
-	test1Name              = "test-one"
-	test2Name              = "test-two"
-	testPickerName         = "test-picker"
+	testProfileHandlerType = "test-profile-handler"
+	test1Type              = "test-one"
+	test2Type              = "test-two"
+	testPickerType         = "test-picker"
 )
 
 func TestLoadConfiguration(t *testing.T) {
@@ -50,21 +55,21 @@ func TestLoadConfiguration(t *testing.T) {
 		Plugins: []configapi.PluginSpec{
 			{
 				Name:       "test1",
-				PluginName: test1Name,
+				Type:       test1Type,
 				Parameters: json.RawMessage("{\"threshold\":10}"),
 			},
 			{
-				Name:       "profileHandler",
-				PluginName: "test-profile-handler",
+				Name: "profileHandler",
+				Type: "test-profile-handler",
 			},
 			{
-				Name:       test2Name,
-				PluginName: test2Name,
+				Name:       test2Type,
+				Type:       test2Type,
 				Parameters: json.RawMessage("{\"hashBlockSize\":32}"),
 			},
 			{
-				Name:       "testPicker",
-				PluginName: testPickerName,
+				Name: "testPicker",
+				Type: testPickerType,
 			},
 		},
 		SchedulingProfiles: []configapi.SchedulingProfile{
@@ -154,12 +159,6 @@ func TestLoadConfiguration(t *testing.T) {
 			configFile: "",
 			wantErr:    true,
 		},
-		{
-			name:       "errorBadProfilePluginName",
-			configText: errorBadProfilePluginNameText,
-			configFile: "",
-			wantErr:    true,
-		},
 		{
 			name:       "errorDuplicatePlugin",
 			configText: errorDuplicatePluginText,
@@ -175,14 +174,14 @@ func TestLoadConfiguration(t *testing.T) {
 		{
 			name:       "successFromFile",
 			configText: "",
-			configFile: "../../../../test/testdata/configloader_1_test.yaml",
+			configFile: "../../../../../test/testdata/configloader_1_test.yaml",
 			want:       goodConfig,
 			wantErr:    false,
 		},
 		{
 			name:       "noSuchFile",
 			configText: "",
-			configFile: "../../../../test/testdata/configloader_error_test.yaml",
+			configFile: "../../../../../test/testdata/configloader_error_test.yaml",
 			wantErr:    true,
 		},
 	}
@@ -210,14 +209,15 @@ func TestLoadPluginReferences(t *testing.T) {
 	if err != nil {
 		t.Fatalf("LoadConfig returned unexpected error: %v", err)
 	}
-	references, err := LoadPluginReferences(theConfig.Plugins, testHandle{})
+	handle := utils.NewTestHandle()
+	err = LoadPluginReferences(theConfig.Plugins, handle)
 	if err != nil {
 		t.Fatalf("LoadPluginReferences returned unexpected error: %v", err)
 	}
-	if len(references) == 0 {
+	if len(handle.Plugins().GetAllPlugins()) == 0 {
 		t.Fatalf("LoadPluginReferences returned an empty set of references")
 	}
-	if t1, ok := references["test1"]; !ok {
+	if t1 := handle.Plugins().Plugin("test1"); t1 == nil {
 		t.Fatalf("LoadPluginReferences returned references did not contain test1")
 	} else if _, ok := t1.(*test1); !ok {
 		t.Fatalf("LoadPluginReferences returned references value for test1 has the wrong type %#v", t1)
@@ -227,41 +227,121 @@ func TestLoadPluginReferences(t *testing.T) {
 	if err != nil {
 		t.Fatalf("LoadConfig returned unexpected error: %v", err)
 	}
-	_, err = LoadPluginReferences(theConfig.Plugins, testHandle{})
+	err = LoadPluginReferences(theConfig.Plugins, utils.NewTestHandle())
 	if err == nil {
 		t.Fatalf("LoadPluginReferences did not return the expected error")
 	}
 }
 
 func TestInstantiatePlugin(t *testing.T) {
-	plugSpec := configapi.PluginSpec{PluginName: "plover"}
-	_, err := InstantiatePlugin(plugSpec, testHandle{})
+	plugSpec := configapi.PluginSpec{Type: "plover"}
+	_, err := instantiatePlugin(plugSpec, utils.NewTestHandle())
 	if err == nil {
 		t.Fatalf("InstantiatePlugin did not return the expected error")
 	}
 }
 
-type testHandle struct {
+func TestLoadSchedulerConfig(t *testing.T) {
+	tests := []struct {
+		name       string
+		configText string
+		wantErr    bool
+	}{
+		{
+			name:       "schedulerSuccess",
+			configText: successSchedulerConfigText,
+			wantErr:    false,
+		},
+		{
+			name:       "errorBadPluginJson",
+			configText: errorBadPluginJsonText,
+			wantErr:    true,
+		},
+		{
+			name:       "errorBadReferenceNoWeight",
+			configText: errorBadReferenceNoWeightText,
+			wantErr:    true,
+		},
+		{
+			name:       "errorTwoPickers",
+			configText: errorTwoPickersText,
+			wantErr:    true,
+		},
+		{
+			name:       "errorConfig",
+			configText: errorConfigText,
+			wantErr:    true,
+		},
+		{
+			name:       "errorTwoProfileHandlers",
+			configText: errorTwoProfileHandlersText,
+			wantErr:    true,
+		},
+		{
+			name:       "errorNoProfileHandlers",
+			configText: errorNoProfileHandlersText,
+			wantErr:    true,
+		},
+	}
+
+	registerNeededPlgugins()
+
+	for _, test := range tests {
+		theConfig, err := LoadConfig([]byte(test.configText), "")
+		if err != nil {
+			if test.wantErr {
+				continue
+			}
+			t.Fatalf("LoadConfig returned unexpected error: %v", err)
+		}
+		handle := utils.NewTestHandle()
+		err = LoadPluginReferences(theConfig.Plugins, handle)
+		if err != nil {
+			if test.wantErr {
+				continue
+			}
+			t.Fatalf("LoadPluginReferences returned unexpected error: %v", err)
+		}
+
+		_, err = LoadSchedulerConfig(theConfig.SchedulingProfiles, handle)
+		if err != nil {
+			if !test.wantErr {
+				t.Errorf("LoadSchedulerConfig returned an unexpected error. error %v", err)
+			}
+		} else if test.wantErr {
+			t.Errorf("LoadSchedulerConfig did not return an expected error (%s)", test.name)
+		}
+	}
+}
+
+func registerNeededPlgugins() {
+	plugins.Register(filter.LowQueueFilterType, filter.LowQueueFilterFactory)
+	plugins.Register(prefix.PrefixCachePluginType, prefix.PrefixCachePluginFactory)
+	plugins.Register(picker.MaxScorePickerType, picker.MaxScorePickerFactory)
+	plugins.Register(picker.RandomPickerType, picker.RandomPickerFactory)
+	plugins.Register(profile.SingleProfileHandlerType, profile.SingleProfileHandlerFactory)
 }
 
 // The following multi-line string constants, cause false positive lint errors (dupword)
 
+// valid configuration
+//
 //nolint:dupword
 const successConfigText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: profileHandler
-  pluginName: test-profile-handler
-- pluginName: test-two
+  type: test-profile-handler
+- type: test-two
   parameters:
     hashBlockSize: 32
 - name: testPicker
-  pluginName: test-picker
+  type: test-picker
 schedulingProfiles:
 - name: default
   plugins:
@@ -271,6 +351,8 @@ schedulingProfiles:
   - pluginRef: testPicker
 `
 
+// YAML does not follow expected structure of config
+//
 //nolint:dupword
 const errorBadYamlText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
@@ -279,6 +361,8 @@ plugins:
 - testing 1 2 3
 `
 
+// missing required Plugin type
+//
 //nolint:dupword
 const errorBadPluginReferenceText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
@@ -288,165 +372,173 @@ plugins:
     a: 1234
 `
 
+// plugin type does not exist
+//
 //nolint:dupword
 const errorBadPluginReferencePluginText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: testx
-  pluginName: test-x
+  type: test-x
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 `
 
+// missing required profile handler
+//
 //nolint:dupword
 const errorNoProfileHandlerText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 schedulingProfiles:
 - name: default
 `
 
+// missing scheduling profiles
+//
 //nolint:dupword
 const errorNoProfilesText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 `
 
+// missing required scheduling profile name
+//
 //nolint:dupword
 const errorNoProfileNameText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - plugins:
   - pluginRef: test1
 `
 
+// missing plugins in scheduling profile
+//
 //nolint:dupword
 const errorNoProfilePluginsText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - name: default
 `
 
+// missing required plugin reference name, only weight is provided
+//
 //nolint:dupword
 const errorBadProfilePluginText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - name: default
   plugins:
   - weight: 10
 `
 
+// reference a non-existent plugin
+//
 //nolint:dupword
 const errorBadProfilePluginRefText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: profileHandler
-  pluginName: test-profile-handler
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: plover
-`
-
-//nolint:dupword
-const errorBadProfilePluginNameText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - name: default
   plugins:
   - pluginRef: plover
 `
 
+// invalid parameters (string provided where int is expected)
+//
 //nolint:dupword
 const errorBadPluginReferenceParametersText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: asdf
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - name: default
   plugins:
   - pluginRef: test1
 `
 
+// duplicate names in plugin list
+//
 //nolint:dupword
 const errorDuplicatePluginText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 20
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - name: default
   plugins:
   - pluginRef: test1
 `
 
+// duplicate scheduling profile name
+//
 //nolint:dupword
 const errorDuplicateProfileText = `
 apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: test2
   pluginName: test-one
-  parameters:
+  type:
     threshold: 20
 - name: profileHandler
-  pluginName: test-profile-handler
+  type: test-profile-handler
 schedulingProfiles:
 - name: default
   plugins:
@@ -463,12 +555,16 @@ type test1 struct {
 	Threshold int `json:"threshold"`
 }
 
+func (f *test1) Type() string {
+	return test1Type
+}
+
 func (f *test1) Name() string {
-	return test1Name
+	return "test-1"
 }
 
 // Filter filters out pods that doesn't meet the filter criteria.
-func (f *test1) Filter(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *test1) Filter(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) []types.Pod {
 	return pods
 }
 
@@ -478,27 +574,34 @@ var _ framework.PostCycle = &test2{}
 
 type test2 struct{}
 
+func (f *test2) Type() string {
+	return test2Type
+}
+
 func (f *test2) Name() string {
-	return test2Name
+	return "test-2"
 }
 
-func (m *test2) Score(ctx context.Context, request *types.LLMRequest, cycleState *types.CycleState, pods []types.Pod) map[types.Pod]float64 {
+func (m *test2) Score(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, _ []types.Pod) map[types.Pod]float64 {
 	return map[types.Pod]float64{}
 }
 
-func (m *test2) PostCycle(ctx context.Context, cycleState *types.CycleState, res *types.ProfileRunResult) {
-}
+func (m *test2) PostCycle(_ context.Context, _ *types.CycleState, _ *types.ProfileRunResult) {}
 
 // compile-time type validation
 var _ framework.Picker = &testPicker{}
 
 type testPicker struct{}
 
+func (p *testPicker) Type() string {
+	return testPickerType
+}
+
 func (p *testPicker) Name() string {
-	return testPickerName
+	return "test-picker"
 }
 
-func (p *testPicker) Pick(ctx context.Context, cycleState *types.CycleState, scoredPods []*types.ScoredPod) *types.ProfileRunResult {
+func (p *testPicker) Pick(_ context.Context, _ *types.CycleState, _ []*types.ScoredPod) *types.ProfileRunResult {
 	return nil
 }
 
@@ -507,20 +610,24 @@ var _ framework.ProfileHandler = &testProfileHandler{}
 
 type testProfileHandler struct{}
 
+func (p *testProfileHandler) Type() string {
+	return testProfileHandlerType
+}
+
 func (p *testProfileHandler) Name() string {
-	return testProfileHandlerName
+	return "test-profile-handler"
 }
 
-func (p *testProfileHandler) Pick(ctx context.Context, request *types.LLMRequest, profiles map[string]*framework.SchedulerProfile, executionResults map[string]*types.ProfileRunResult) map[string]*framework.SchedulerProfile {
+func (p *testProfileHandler) Pick(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, _ map[string]*framework.SchedulerProfile, _ map[string]*types.ProfileRunResult) map[string]*framework.SchedulerProfile {
 	return nil
 }
 
-func (p *testProfileHandler) ProcessResults(ctx context.Context, request *types.LLMRequest, profileResults map[string]*types.ProfileRunResult) *types.SchedulingResult {
-	return nil
+func (p *testProfileHandler) ProcessResults(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, _ map[string]*types.ProfileRunResult) (*types.SchedulingResult, error) {
+	return nil, nil
 }
 
 func registerTestPlugins() {
-	plugins.Register(test1Name,
+	plugins.Register(test1Type,
 		func(name string, parameters json.RawMessage, handle plugins.Handle) (plugins.Plugin, error) {
 			result := test1{}
 			err := json.Unmarshal(parameters, &result)
@@ -528,21 +635,155 @@ func registerTestPlugins() {
 		},
 	)
 
-	plugins.Register(test2Name,
+	plugins.Register(test2Type,
 		func(name string, parameters json.RawMessage, handle plugins.Handle) (plugins.Plugin, error) {
 			return &test2{}, nil
 		},
 	)
 
-	plugins.Register(testPickerName,
+	plugins.Register(testPickerType,
 		func(name string, parameters json.RawMessage, handle plugins.Handle) (plugins.Plugin, error) {
 			return &testPicker{}, nil
 		},
 	)
 
-	plugins.Register(testProfileHandlerName,
+	plugins.Register(testProfileHandlerType,
 		func(name string, parameters json.RawMessage, handle plugins.Handle) (plugins.Plugin, error) {
 			return &testProfileHandler{}, nil
 		},
 	)
 }
+
+// valid configuration
+//
+//nolint:dupword
+const successSchedulerConfigText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: lowQueue
+  type: low-queue
+  parameters:
+    threshold: 10
+- name: prefixCache
+  type: prefix-cache
+  parameters:
+    hashBlockSize: 32
+- name: maxScore
+  type: max-score
+- name: profileHandler
+  type: single-profile
+schedulingProfiles:
+- name: default
+  plugins:
+  - pluginRef: lowQueue
+  - pluginRef: prefixCache
+    weight: 50
+  - pluginRef: maxScore
+`
+
+// invalid parameter configuration for plugin (string passed, in expected)
+//
+//nolint:dupword
+const errorBadPluginJsonText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name:profileHandler
+  type: single-profile
+- name: prefixCache
+  type: prefix-cache
+  parameters:
+    hashBlockSize: asdf
+schedulingProfiles:
+- name: default
+  plugins:
+  - pluginRef: prefixCache
+    weight: 50
+`
+
+// missing weight for scorer
+//
+//nolint:dupword
+const errorBadReferenceNoWeightText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: profileHandler
+  type: single-profile
+- name: prefixCache
+  type: prefix-cache
+  parameters:
+    hashBlockSize: 32
+schedulingProfiles:
+- name: default
+  plugins:
+  - pluginRef: prefixCache
+`
+
+// multiple pickers in scheduling profile
+//
+//nolint:dupword
+const errorTwoPickersText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: profileHandler
+  type: single-profile
+- name: maxScore
+  type: max-score
+- name: random
+  type: random
+schedulingProfiles:
+- name: default
+  plugins:
+  - pluginRef: maxScore
+  - pluginRef: random
+`
+
+// missing required scheduling profile
+//
+//nolint:dupword
+const errorConfigText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: lowQueue
+  pluginName: low-queue
+  parameters:
+    threshold: 10
+`
+
+// multiple profile handlers when only one is allowed
+//
+//nolint:dupword
+const errorTwoProfileHandlersText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: profileHandler
+  type: single-profile
+- name: secondProfileHandler
+  type: single-profile
+- name: maxScore
+  type: max-score
+schedulingProfiles:
+- name: default
+  plugins:
+  - pluginRef: maxScore
+`
+
+// missing required profile handler
+//
+//nolint:dupword
+const errorNoProfileHandlersText = `
+apiVersion: inference.networking.x-k8s.io/v1alpha1
+kind: EndpointPickerConfig
+plugins:
+- name: maxScore
+  type: max-score
+schedulingProfiles:
+- name: default
+  plugins:
+  - pluginRef: maxScore
+`
diff --git a/pkg/epp/requestcontrol/director_test.go b/pkg/epp/requestcontrol/director_test.go
index 9d90461..aa39666 100644
--- a/pkg/epp/requestcontrol/director_test.go
+++ b/pkg/epp/requestcontrol/director_test.go
@@ -58,7 +58,7 @@ type mockScheduler struct {
 	scheduleErr     error
 }
 
-func (m *mockScheduler) Schedule(ctx context.Context, req *schedulingtypes.LLMRequest) (*schedulingtypes.SchedulingResult, error) {
+func (m *mockScheduler) Schedule(_ context.Context, _ *schedulingtypes.LLMRequest, _ []schedulingtypes.Pod) (*schedulingtypes.SchedulingResult, error) {
 	return m.scheduleResults, m.scheduleErr
 }
 
@@ -268,6 +268,27 @@ func TestDirector_HandleRequest(t *testing.T) {
 			},
 			wantMutatedBodyModel: "resolved-target-model-A",
 		},
+		{
+			name: "nonexistent target defined, use default inference model",
+			schedulerMockSetup: func(m *mockScheduler) {
+				m.scheduleResults = defaultSuccessfulScheduleResults
+			},
+			wantReqCtx: &handlers.RequestContext{
+				Model:               "food-review-1",
+				ResolvedTargetModel: "food-review-1",
+				TargetPod: &backend.Pod{
+					NamespacedName: types.NamespacedName{Namespace: "default", Name: "pod1"},
+					Address:        "192.168.1.100",
+				},
+				TargetEndpoint: "192.168.1.100:8000",
+			},
+			wantMutatedBodyModel: "food-review-1",
+			reqBodyMap: map[string]interface{}{
+				"model":  "food-review-1",
+				"prompt": "test prompt",
+			},
+			mockSaturationDetector: &mockSaturationDetector{isSaturated: false},
+		},
 		{
 
 			name: "request dropped (sheddable, saturated)",
@@ -298,24 +319,6 @@ func TestDirector_HandleRequest(t *testing.T) {
 			},
 			wantErrCode: errutil.BadRequest,
 		},
-		{
-			name: "invalid model defined, expect err",
-			reqBodyMap: map[string]interface{}{
-				"model":  "non-existent-model",
-				"prompt": "test prompt",
-			},
-			mockSaturationDetector: &mockSaturationDetector{isSaturated: false},
-			wantErrCode:            errutil.BadConfiguration,
-		},
-		{
-			name: "invalid target defined, expect err",
-			reqBodyMap: map[string]interface{}{
-				"model":  "food-review-1",
-				"prompt": "test prompt",
-			},
-			mockSaturationDetector: &mockSaturationDetector{isSaturated: false},
-			wantErrCode:            errutil.BadConfiguration,
-		},
 		{
 			name: "scheduler returns error",
 			reqBodyMap: map[string]interface{}{
@@ -518,7 +521,7 @@ func pointer(v int32) *int32 {
 
 func TestDirector_HandleResponse(t *testing.T) {
 	pr1 := &testPostResponse{
-		NameRes: "pr1",
+		TypeRes: "pr1",
 	}
 
 	ctx := logutil.NewTestLoggerIntoContext(context.Background())
@@ -556,12 +559,13 @@ func TestDirector_HandleResponse(t *testing.T) {
 }
 
 type testPostResponse struct {
-	NameRes                 string
+	TypeRes                 string
 	lastRespOnResponse      *Response
 	lastTargetPodOnResponse string
 }
 
-func (p *testPostResponse) Name() string { return p.NameRes }
+func (p *testPostResponse) Type() string { return p.TypeRes }
+func (p *testPostResponse) Name() string { return "test-post-response" }
 
 func (p *testPostResponse) PostResponse(_ context.Context, _ *schedulingtypes.LLMRequest, response *Response, targetPod *backend.Pod) {
 	p.lastRespOnResponse = response
diff --git a/pkg/epp/scheduling/framework/plugins/filter/filter_test.go b/pkg/epp/scheduling/framework/plugins/filter/filter_test.go
index 27f6943..464936e 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/filter_test.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/filter_test.go
@@ -35,11 +35,15 @@ var _ framework.Filter = &filterAll{}
 
 type filterAll struct{}
 
+func (f *filterAll) Type() string {
+	return "filter-all"
+}
+
 func (f *filterAll) Name() string {
-	return "filter all"
+	return "test-all"
 }
 
-func (f *filterAll) Filter(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (f *filterAll) Filter(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) []types.Pod {
 	return []types.Pod{}
 }
 
@@ -138,7 +142,7 @@ func TestFilter(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			got := test.filter.Filter(context.Background(), test.req, types.NewCycleState(), test.input)
+			got := test.filter.Filter(context.Background(), types.NewCycleState(), test.req, test.input)
 
 			if diff := cmp.Diff(test.output, got); diff != "" {
 				t.Errorf("Unexpected output (-want +got): %v", diff)
@@ -203,10 +207,10 @@ func TestLoRASoftAffinityDistribution(t *testing.T) {
 	expectedAvailabilityPercent := 100 - expectedAffinityPercent
 
 	// initialize LoraAffinityFilter
-	LoraAffinityFilter := NewLoraAffinityFilter()
+	LoraAffinityFilter := NewLoraAffinityFilter(config.Conf.LoraAffinityThreshold)
 
 	for range numIterations {
-		result := LoraAffinityFilter.Filter(context.Background(), req, types.NewCycleState(), pods)
+		result := LoraAffinityFilter.Filter(context.Background(), types.NewCycleState(), req, pods)
 
 		// Check which type of pod was returned
 		if len(result) != 1 {
diff --git a/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin_test.go b/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin_test.go
index db1feac..73f02c3 100644
--- a/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin_test.go
+++ b/pkg/epp/scheduling/framework/plugins/multi/prefix/plugin_test.go
@@ -49,7 +49,7 @@ func TestPrefixPlugin(t *testing.T) {
 		Prompt:      "aaaaaa",
 	}
 	cycleState1 := types.NewCycleState()
-	scores := plugin.Score(context.Background(), req1, cycleState1, pods)
+	scores := plugin.Score(context.Background(), cycleState1, req1, pods)
 	state, err := plugin.getPrefixState(cycleState1)
 	assert.NoError(t, err)
 	t.Logf("Hashes %+v, cached servers: %+v", state.PrefixHashes, state.PrefixCacheServers)
@@ -70,7 +70,7 @@ func TestPrefixPlugin(t *testing.T) {
 		Prompt:      "bbbbbb",
 	}
 	cycleState2 := types.NewCycleState()
-	scores = plugin.Score(context.Background(), req2, cycleState2, pods)
+	scores = plugin.Score(context.Background(), cycleState2, req2, pods)
 	state, err = plugin.getPrefixState(cycleState2)
 	assert.NoError(t, err)
 	t.Logf("Hashes %+v, cached servers: %+v", state.PrefixHashes, state.PrefixCacheServers)
@@ -90,7 +90,7 @@ func TestPrefixPlugin(t *testing.T) {
 		Prompt:      "aaaabbbb",
 	}
 	cycleState3 := types.NewCycleState()
-	scores = plugin.Score(context.Background(), req3, cycleState3, pods)
+	scores = plugin.Score(context.Background(), cycleState3, req3, pods)
 	state, err = plugin.getPrefixState(cycleState3)
 	assert.NoError(t, err)
 	t.Logf("Hashes %+v, cached servers: %+v", state.PrefixHashes, state.PrefixCacheServers)
@@ -109,7 +109,7 @@ func TestPrefixPlugin(t *testing.T) {
 		Prompt:      "aaaabbbb",
 	}
 	cycleState4 := types.NewCycleState()
-	scores = plugin.Score(context.Background(), req4, cycleState4, pods)
+	scores = plugin.Score(context.Background(), cycleState4, req4, pods)
 	state, err = plugin.getPrefixState(cycleState4)
 	assert.NoError(t, err)
 	t.Logf("Hashes %+v, cached servers: %+v", state.PrefixHashes, state.PrefixCacheServers)
@@ -128,7 +128,7 @@ func TestPrefixPlugin(t *testing.T) {
 		Prompt:      "aaaabbbbcccc",
 	}
 	cycleState5 := types.NewCycleState()
-	scores = plugin.Score(context.Background(), req5, cycleState5, pods)
+	scores = plugin.Score(context.Background(), cycleState5, req5, pods)
 	state, err = plugin.getPrefixState(cycleState5)
 	assert.NoError(t, err)
 	t.Logf("Hashes %+v, cached servers: %+v", state.PrefixHashes, state.PrefixCacheServers)
@@ -179,7 +179,7 @@ func BenchmarkPrefixPluginStress(b *testing.B) {
 
 		// First cycle: simulate scheduling and insert prefix info into the cache
 		cycleState := types.NewCycleState()
-		plugin.Score(context.Background(), req, cycleState, pods)
+		plugin.Score(context.Background(), cycleState, req, pods)
 		plugin.PostCycle(context.Background(), cycleState, &types.ProfileRunResult{TargetPod: pod})
 
 		// Second cycle: validate internal state
diff --git a/pkg/epp/scheduling/framework/plugins/scorer/kvcache_test.go b/pkg/epp/scheduling/framework/plugins/scorer/kvcache_test.go
index 7ed4e3c..c0eeb52 100644
--- a/pkg/epp/scheduling/framework/plugins/scorer/kvcache_test.go
+++ b/pkg/epp/scheduling/framework/plugins/scorer/kvcache_test.go
@@ -83,7 +83,7 @@ func TestKvCacheScorer(t *testing.T) {
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
 			scorer := &KVCacheScorer{}
-			scores := scorer.Score(context.Background(), &types.LLMRequest{}, types.NewCycleState(), test.pods)
+			scores := scorer.Score(context.Background(), types.NewCycleState(), &types.LLMRequest{}, test.pods)
 
 			for i, pod := range test.pods {
 				expectedScore := test.expectedScoresPod[i]
diff --git a/pkg/epp/scheduling/framework/plugins/scorer/queue_test.go b/pkg/epp/scheduling/framework/plugins/scorer/queue_test.go
index 70422ff..a9a8115 100644
--- a/pkg/epp/scheduling/framework/plugins/scorer/queue_test.go
+++ b/pkg/epp/scheduling/framework/plugins/scorer/queue_test.go
@@ -73,7 +73,7 @@ func TestQueueScorer(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			scores := scorer.Score(context.Background(), &types.LLMRequest{}, types.NewCycleState(), test.pods)
+			scores := scorer.Score(context.Background(), types.NewCycleState(), &types.LLMRequest{}, test.pods)
 
 			for i, pod := range test.pods {
 				expectedScore := test.expectedScoresPod[i]
diff --git a/pkg/epp/scheduling/framework/scheduler_profile_test.go b/pkg/epp/scheduling/framework/scheduler_profile_test.go
index fcc877c..d94bb26 100644
--- a/pkg/epp/scheduling/framework/scheduler_profile_test.go
+++ b/pkg/epp/scheduling/framework/scheduler_profile_test.go
@@ -30,21 +30,21 @@ import (
 
 func TestSchedulePlugins(t *testing.T) {
 	tp1 := &testPlugin{
-		NameRes:   "test1",
+		TypeRes:   "test1",
 		ScoreRes:  0.3,
 		FilterRes: []k8stypes.NamespacedName{{Name: "pod1"}, {Name: "pod2"}, {Name: "pod3"}},
 	}
 	tp2 := &testPlugin{
-		NameRes:   "test2",
+		TypeRes:   "test2",
 		ScoreRes:  0.8,
 		FilterRes: []k8stypes.NamespacedName{{Name: "pod1"}, {Name: "pod2"}},
 	}
 	tp_filterAll := &testPlugin{
-		NameRes:   "filter all",
+		TypeRes:   "filter all",
 		FilterRes: []k8stypes.NamespacedName{},
 	}
 	pickerPlugin := &testPlugin{
-		NameRes: "picker",
+		TypeRes: "picker",
 		PickRes: k8stypes.NamespacedName{Name: "pod1"},
 	}
 
@@ -155,24 +155,24 @@ func TestSchedulePlugins(t *testing.T) {
 			for _, plugin := range test.profile.filters {
 				tp, _ := plugin.(*testPlugin)
 				if tp.FilterCallCount != 1 {
-					t.Errorf("Plugin %s Filter() called %d times, expected 1", plugin.Name(), tp.FilterCallCount)
+					t.Errorf("Plugin %s Filter() called %d times, expected 1", plugin.Type(), tp.FilterCallCount)
 				}
 			}
 			for _, plugin := range test.profile.scorers {
 				tp, _ := plugin.Scorer.(*testPlugin)
 				if tp.ScoreCallCount != 1 {
-					t.Errorf("Plugin %s Score() called %d times, expected 1", plugin.Name(), tp.ScoreCallCount)
+					t.Errorf("Plugin %s Score() called %d times, expected 1", plugin.Type(), tp.ScoreCallCount)
 				}
 				if test.numPodsToScore != tp.NumOfScoredPods {
-					t.Errorf("Plugin %s Score() called with %d pods, expected %d", plugin.Name(), tp.NumOfScoredPods, test.numPodsToScore)
+					t.Errorf("Plugin %s Score() called with %d pods, expected %d", plugin.Type(), tp.NumOfScoredPods, test.numPodsToScore)
 				}
 			}
 			tp, _ := test.profile.picker.(*testPlugin)
 			if tp.NumOfPickerCandidates != test.numPodsToScore {
-				t.Errorf("Picker plugin %s Pick() called with %d candidates, expected %d", tp.Name(), tp.NumOfPickerCandidates, tp.NumOfScoredPods)
+				t.Errorf("Picker plugin %s Pick() called with %d candidates, expected %d", tp.Type(), tp.NumOfPickerCandidates, tp.NumOfScoredPods)
 			}
 			if tp.PickCallCount != 1 {
-				t.Errorf("Picker plugin %s Pick() called %d times, expected 1", tp.Name(), tp.PickCallCount)
+				t.Errorf("Picker plugin %s Pick() called %d times, expected 1", tp.Type(), tp.PickCallCount)
 			}
 			if tp.WinnderPodScore != test.targetPodScore {
 				t.Errorf("winnder pod score %v, expected %v", tp.WinnderPodScore, test.targetPodScore)
@@ -180,7 +180,7 @@ func TestSchedulePlugins(t *testing.T) {
 			for _, plugin := range test.profile.postCyclePlugins {
 				tp, _ := plugin.(*testPlugin)
 				if tp.PostScheduleCallCount != 1 {
-					t.Errorf("Plugin %s PostSchedule() called %d times, expected 1", plugin.Name(), tp.PostScheduleCallCount)
+					t.Errorf("Plugin %s PostSchedule() called %d times, expected 1", plugin.Type(), tp.PostScheduleCallCount)
 				}
 			}
 		})
@@ -195,7 +195,7 @@ var _ PostCycle = &testPlugin{}
 
 // testPlugin is an implementation useful in unit tests.
 type testPlugin struct {
-	NameRes               string
+	TypeRes               string
 	ScoreCallCount        int
 	NumOfScoredPods       int
 	ScoreRes              float64
@@ -208,15 +208,16 @@ type testPlugin struct {
 	WinnderPodScore       float64
 }
 
-func (tp *testPlugin) Name() string { return tp.NameRes }
+func (tp *testPlugin) Type() string { return tp.TypeRes }
+func (tp *testPlugin) Name() string { return "test-plugin" }
 
-func (tp *testPlugin) Filter(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) []types.Pod {
+func (tp *testPlugin) Filter(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) []types.Pod {
 	tp.FilterCallCount++
 	return findPods(pods, tp.FilterRes...)
 
 }
 
-func (tp *testPlugin) Score(_ context.Context, _ *types.LLMRequest, _ *types.CycleState, pods []types.Pod) map[types.Pod]float64 {
+func (tp *testPlugin) Score(_ context.Context, _ *types.CycleState, _ *types.LLMRequest, pods []types.Pod) map[types.Pod]float64 {
 	tp.ScoreCallCount++
 	scoredPods := make(map[types.Pod]float64, len(pods))
 	for _, pod := range pods {
diff --git a/pkg/epp/scheduling/scheduler_config_test.go b/pkg/epp/scheduling/scheduler_config_test.go
deleted file mode 100644
index 495df31..0000000
--- a/pkg/epp/scheduling/scheduler_config_test.go
+++ /dev/null
@@ -1,263 +0,0 @@
-/*
-Copyright 2025 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package scheduling
-
-import (
-	"testing"
-
-	commonconfig "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/filter"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/multi/prefix"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
-	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
-)
-
-func TestLoadSchedulerConfig(t *testing.T) {
-	log := logutil.NewTestLogger()
-
-	tests := []struct {
-		name       string
-		configText string
-		wantErr    bool
-	}{
-		{
-			name:       "success",
-			configText: successConfigText,
-			wantErr:    false,
-		},
-		{
-			name:       "errorBadPluginJson",
-			configText: errorBadPluginJsonText,
-			wantErr:    true,
-		},
-		{
-			name:       "errorBadReferenceNoWeight",
-			configText: errorBadReferenceNoWeightText,
-			wantErr:    true,
-		},
-		{
-			name:       "errorPluginReferenceJson",
-			configText: errorPluginReferenceJsonText,
-			wantErr:    true,
-		},
-		{
-			name:       "errorTwoPickers",
-			configText: errorTwoPickersText,
-			wantErr:    true,
-		},
-		{
-			name:       "errorConfig",
-			configText: errorConfigText,
-			wantErr:    true,
-		},
-		{
-			name:       "errorTwoProfileHandlers",
-			configText: errorTwoProfileHandlersText,
-			wantErr:    true,
-		},
-		{
-			name:       "errorNoProfileHandlers",
-			configText: errorNoProfileHandlersText,
-			wantErr:    true,
-		},
-	}
-
-	registerNeededPlgugins()
-
-	for _, test := range tests {
-		theConfig, err := commonconfig.LoadConfig([]byte(test.configText), "")
-		if err != nil {
-			if test.wantErr {
-				continue
-			}
-			t.Fatalf("LoadConfig returned unexpected error: %v", err)
-		}
-		instantiatedPlugins, err := commonconfig.LoadPluginReferences(theConfig.Plugins, testHandle{})
-		if err != nil {
-			if test.wantErr {
-				continue
-			}
-			t.Fatalf("LoadPluginReferences returned unexpected error: %v", err)
-		}
-
-		_, err = LoadSchedulerConfig(theConfig.SchedulingProfiles, instantiatedPlugins, log)
-		if err != nil {
-			if !test.wantErr {
-				t.Errorf("LoadSchedulerConfig returned an unexpected error. error %v", err)
-			}
-		} else if test.wantErr {
-			t.Errorf("LoadSchedulerConfig did not return an expected error (%s)", test.name)
-		}
-	}
-}
-
-type testHandle struct {
-}
-
-func registerNeededPlgugins() {
-	allPlugins := map[string]plugins.Factory{
-		filter.LowQueueFilterName:        filter.LowQueueFilterFactory,
-		prefix.PrefixCachePluginName:     prefix.PrefixCachePluginFactory,
-		picker.MaxScorePickerName:        picker.MaxScorePickerFactory,
-		picker.RandomPickerName:          picker.RandomPickerFactory,
-		profile.SingleProfileHandlerName: profile.SingleProfileHandlerFactory,
-	}
-	for name, factory := range allPlugins {
-		plugins.Register(name, factory)
-	}
-}
-
-// The following multi-line string constants, cause false positive lint errors (dupword)
-
-//nolint:dupword
-const successConfigText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: lowQueue
-  pluginName: low-queue
-  parameters:
-    threshold: 10
-- name: prefixCache
-  pluginName: prefix-cache
-  parameters:
-    hashBlockSize: 32
-- name: maxScore
-  pluginName: max-score
-- name: profileHandler
-  pluginName: single-profile
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: lowQueue
-  - pluginRef: prefixCache
-    weight: 50
-  - pluginRef: maxScore
-`
-
-//nolint:dupword
-const errorBadPluginJsonText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name:profileHandler
-  pluginName: single-profile
-- name: prefixCache
-  pluginName: prefix-cache
-  parameters:
-    hashBlockSize: asdf
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: prefixCache
-    weight: 50
-`
-
-//nolint:dupword
-const errorBadReferenceNoWeightText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: profileHandler
-  pluginName: single-profile
-- name: prefixCache
-  pluginName: prefix-cache
-  parameters:
-    hashBlockSize: 32
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: prefixCache
-`
-
-//nolint:dupword
-const errorPluginReferenceJsonText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: lowQueue
-  pluginName: low-queue
-  parameters:
-    threshold: qwer
-- name: profileHandler
-  pluginName: single-profile
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: lowQueue
-`
-
-//nolint:dupword
-const errorTwoPickersText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: profileHandler
-  pluginName: single-profile
-- name: maxScore
-  pluginName: max-score
-- name: random
-  pluginName: random
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: maxScore
-  - pluginRef: random
-`
-
-//nolint:dupword
-const errorConfigText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: lowQueue
-  pluginName: low-queue
-  parameters:
-    threshold: 10
-`
-
-//nolint:dupword
-const errorTwoProfileHandlersText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: profileHandler
-  pluginName: single-profile
-- name: secondProfileHandler
-  pluginName: single-profile
-- name: maxScore
-  pluginName: max-score
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: maxScore
-`
-
-//nolint:dupword
-const errorNoProfileHandlersText = `
-apiVersion: inference.networking.x-k8s.io/v1alpha1
-kind: EndpointPickerConfig
-plugins:
-- name: maxScore
-  pluginName: max-score
-schedulingProfiles:
-- name: default
-  plugins:
-  - pluginRef: maxScore
-`
diff --git a/pkg/epp/scheduling/scheduler_test.go b/pkg/epp/scheduling/scheduler_test.go
index 3fcbcc6..720a6b4 100644
--- a/pkg/epp/scheduling/scheduler_test.go
+++ b/pkg/epp/scheduling/scheduler_test.go
@@ -33,17 +33,17 @@ func TestSchedule(t *testing.T) {
 	tests := []struct {
 		name    string
 		req     *types.LLMRequest
-		input   []*backendmetrics.FakePodMetrics
+		input   []backendmetrics.PodMetrics
 		wantRes *types.SchedulingResult
 		err     bool
 	}{
 		{
-			name: "no pods in datastore",
+			name: "no candidate pods",
 			req: &types.LLMRequest{
 				TargetModel: "any-model",
 				RequestId:   uuid.NewString(),
 			},
-			input:   []*backendmetrics.FakePodMetrics{},
+			input:   []backendmetrics.PodMetrics{},
 			wantRes: nil,
 			err:     true,
 		},
@@ -55,8 +55,8 @@ func TestSchedule(t *testing.T) {
 			},
 			// pod2 will be picked because it has relatively low queue size, with the requested
 			// model being active, and has low KV cache.
-			input: []*backendmetrics.FakePodMetrics{
-				{
+			input: []backendmetrics.PodMetrics{
+				&backendmetrics.FakePodMetrics{
 					Pod: &backend.Pod{NamespacedName: k8stypes.NamespacedName{Name: "pod1"}},
 					Metrics: &backendmetrics.MetricsState{
 						WaitingQueueSize:    0,
@@ -68,7 +68,7 @@ func TestSchedule(t *testing.T) {
 						},
 					},
 				},
-				{
+				&backendmetrics.FakePodMetrics{
 					Pod: &backend.Pod{NamespacedName: k8stypes.NamespacedName{Name: "pod2"}},
 					Metrics: &backendmetrics.MetricsState{
 						WaitingQueueSize:    3,
@@ -80,7 +80,7 @@ func TestSchedule(t *testing.T) {
 						},
 					},
 				},
-				{
+				&backendmetrics.FakePodMetrics{
 					Pod: &backend.Pod{NamespacedName: k8stypes.NamespacedName{Name: "pod3"}},
 					Metrics: &backendmetrics.MetricsState{
 						WaitingQueueSize:    10,
@@ -119,8 +119,8 @@ func TestSchedule(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			scheduler := NewScheduler(&fakeDataStore{pods: test.input})
-			got, err := scheduler.Schedule(context.Background(), test.req)
+			scheduler := NewScheduler()
+			got, err := scheduler.Schedule(context.Background(), test.req, types.ToSchedulerPodMetrics(test.input))
 			if test.err != (err != nil) {
 				t.Errorf("Unexpected error, got %v, want %v", err, test.err)
 			}
@@ -131,15 +131,3 @@ func TestSchedule(t *testing.T) {
 		})
 	}
 }
-
-type fakeDataStore struct {
-	pods []*backendmetrics.FakePodMetrics
-}
-
-func (fds *fakeDataStore) PodGetAll() []backendmetrics.PodMetrics {
-	pm := make([]backendmetrics.PodMetrics, 0, len(fds.pods))
-	for _, pod := range fds.pods {
-		pm = append(pm, pod)
-	}
-	return pm
-}
diff --git a/test/e2e/epp/e2e_suite_test.go b/test/e2e/epp/e2e_suite_test.go
index 6595923..04f4e89 100644
--- a/test/e2e/epp/e2e_suite_test.go
+++ b/test/e2e/epp/e2e_suite_test.go
@@ -18,6 +18,7 @@ package epp
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"os"
 	"strings"
@@ -70,6 +71,8 @@ const (
 	envoyPort = "8081"
 	// inferExtName is the name of the inference extension test resources.
 	inferExtName = "vllm-llama3-8b-instruct-epp"
+	// metricsReaderSecretName is the name of the metrics reader secret which stores sa token to read epp metrics.
+	metricsReaderSecretName = "inference-gateway-sa-metrics-reader-secret"
 	// clientManifest is the manifest for the client test resources.
 	clientManifest = "../../testdata/client.yaml"
 	// modelServerSecretManifest is the manifest for the model server secret resource.
@@ -82,6 +85,8 @@ const (
 	inferExtManifest = "../../testdata/inferencepool-e2e.yaml"
 	// envoyManifest is the manifest for the envoy proxy test resources.
 	envoyManifest = "../../testdata/envoy.yaml"
+	// metricsRbacManifest is the manifest for the rbac resources for testing metrics.
+	metricsRbacManifest = "../../testdata/metrics-rbac.yaml"
 	// modelServerManifestFilepathEnvVar is the env var that holds absolute path to the manifest for the model server test resource.
 	modelServerManifestFilepathEnvVar = "MANIFEST_PATH"
 )
@@ -133,6 +138,7 @@ func setupInfra() {
 	createInferExt(cli, inferExtManifest)
 	createClient(cli, clientManifest)
 	createEnvoy(cli, envoyManifest)
+	createMetricsRbac(cli, metricsRbacManifest)
 	// Run this step last, as it requires additional time for the model server to become ready.
 	createModelServer(cli, modelServerManifestArray, modelServerManifestPath)
 }
@@ -259,6 +265,30 @@ func createClient(k8sClient client.Client, filePath string) {
 	testutils.PodReady(ctx, k8sClient, pod, readyTimeout, interval)
 }
 
+// createMetricsRbac creates the metrics RBAC resources from the manifest file.
+func createMetricsRbac(k8sClient client.Client, filePath string) {
+	inManifests := readYaml(filePath)
+	ginkgo.By("Replacing placeholder namespace with E2E_NS environment variable")
+	outManifests := []string{}
+	for _, m := range inManifests {
+		outManifests = append(outManifests, strings.ReplaceAll(m, "$E2E_NS", nsName))
+	}
+	ginkgo.By("Creating RBAC resources for scraping metrics from manifest: " + filePath)
+	createObjsFromYaml(k8sClient, outManifests)
+
+	// wait for sa token to exist
+	testutils.EventuallyExists(ctx, func() error {
+		token, err := getMetricsReaderToken(k8sClient)
+		if err != nil {
+			return err
+		}
+		if len(token) == 0 {
+			return errors.New("failed to get metrics reader token")
+		}
+		return nil
+	}, existsTimeout, interval)
+}
+
 // createModelServer creates the model server resources used for testing from the given filePaths.
 func createModelServer(k8sClient client.Client, modelServerManifestArray []string, deployPath string) {
 	ginkgo.By("Creating model server resources from manifest: " + deployPath)
diff --git a/test/e2e/epp/e2e_test.go b/test/e2e/epp/e2e_test.go
index b1ba47a..7bf31ac 100644
--- a/test/e2e/epp/e2e_test.go
+++ b/test/e2e/epp/e2e_test.go
@@ -17,6 +17,8 @@ limitations under the License.
 package epp
 
 import (
+	"encoding/json"
+	"errors"
 	"fmt"
 	"strconv"
 	"strings"
@@ -26,9 +28,12 @@ import (
 	"github.com/google/go-cmp/cmp/cmpopts"
 	"github.com/onsi/ginkgo/v2"
 	"github.com/onsi/gomega"
+	corev1 "k8s.io/api/core/v1"
+	k8serrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/utils/ptr"
-	"sigs.k8s.io/gateway-api-inference-extension/api/v1alpha2"
+	client "sigs.k8s.io/controller-runtime/pkg/client"
+	v1alpha2 "sigs.k8s.io/gateway-api-inference-extension/api/v1alpha2"
 	testutils "sigs.k8s.io/gateway-api-inference-extension/test/utils"
 )
 
@@ -51,38 +56,57 @@ var _ = ginkgo.Describe("InferencePool", func() {
 	ginkgo.AfterEach(func() {
 		ginkgo.By("Deleting the InferenceModel test resource.")
 		cleanupInferModelResources()
+		gomega.Eventually(func() error {
+			err := cli.Get(ctx, types.NamespacedName{Namespace: infModel.Namespace, Name: infModel.Name}, infModel)
+			if err == nil {
+				return errors.New("InferenceModel resource still exists")
+			}
+			if !k8serrors.IsNotFound(err) {
+				return nil
+			}
+			return nil
+		}, existsTimeout, interval).Should(gomega.Succeed())
 	})
 
 	ginkgo.When("The Inference Extension is running", func() {
 		ginkgo.It("Should route traffic to target model servers", func() {
 			for _, t := range []struct {
 				api              string
-				promptOrMessages string
+				promptOrMessages any
 			}{
 				{
 					api:              "/completions",
 					promptOrMessages: "Write as if you were a critic: San Francisco",
 				},
 				{
-					api:              "/chat/completions",
-					promptOrMessages: `[{"role": "user", "content": "Write as if you were a critic: San Francisco"}]`,
+					api: "/chat/completions",
+					promptOrMessages: []map[string]any{
+						{
+							"role":    "user",
+							"content": "Write as if you were a critic: San Francisco",
+						},
+					},
 				},
 				{
 					api: "/chat/completions",
-					promptOrMessages: `[{"role": "user", "content": "Write as if you were a critic: San Francisco"},` +
-						`{"role": "assistant", "content": "Okay, let's see..."},` +
-						`{"role": "user", "content": "Now summarize your thoughts."}]`,
+					promptOrMessages: []map[string]any{
+						{
+							"role":    "user",
+							"content": "Write as if you were a critic: San Francisco",
+						},
+						{"role": "assistant", "content": "Okay, let's see..."},
+						{"role": "user", "content": "Now summarize your thoughts."},
+					},
 				},
 			} {
-				ginkgo.By("Verifying connectivity through the inference extension with " +
-					t.api + " api and prompt/messages: " + t.promptOrMessages)
+				ginkgo.By(fmt.Sprintf("Verifying connectivity through the inference extension with %s api and prompt/messages: %v", t.api, t.promptOrMessages))
 
 				// Ensure the expected responses include the inferencemodel target model names.
 				var expected []string
 				for _, m := range infModel.Spec.TargetModels {
 					expected = append(expected, m.Name)
 				}
-				curlCmd := getCurlCommand(envoyName, nsName, envoyPort, modelName, curlTimeout, t.api, t.promptOrMessages)
+				curlCmd := getCurlCommand(envoyName, nsName, envoyPort, modelName, curlTimeout, t.api, t.promptOrMessages, false)
 
 				actual := make(map[string]int)
 				gomega.Eventually(func() error {
@@ -106,11 +130,103 @@ var _ = ginkgo.Describe("InferencePool", func() {
 					if !cmp.Equal(got, expected, cmpopts.SortSlices(func(a, b string) bool { return a < b })) {
 						return fmt.Errorf("actual (%v) != expected (%v); resp=%q", got, expected, resp)
 					}
-
 					return nil
 				}, readyTimeout, curlInterval).Should(gomega.Succeed())
 			}
 		})
+
+		ginkgo.It("Should expose EPP metrics after generating traffic", func() {
+			// Define the metrics we expect to see
+			expectedMetrics := []string{
+				"inference_model_request_total",
+				"inference_model_request_error_total",
+				"inference_model_request_duration_seconds",
+				// TODO: normalized_time_per_output_token_seconds is not actually recorded yet
+				// "normalized_time_per_output_token_seconds",
+				"inference_model_request_sizes",
+				"inference_model_response_sizes",
+				"inference_model_input_tokens",
+				"inference_model_output_tokens",
+				"inference_pool_average_kv_cache_utilization",
+				"inference_pool_average_queue_size",
+				"inference_pool_per_pod_queue_size",
+				"inference_model_running_requests",
+				"inference_pool_ready_pods",
+				"inference_extension_info",
+			}
+
+			// Generate traffic by sending requests through the inference extension
+			ginkgo.By("Generating traffic through the inference extension")
+			curlCmd := getCurlCommand(envoyName, nsName, envoyPort, modelName, curlTimeout, "/completions", "Write as if you were a critic: San Francisco", true)
+
+			// Run the curl command multiple times to generate some metrics data
+			for i := 0; i < 5; i++ {
+				_, err := testutils.ExecCommandInPod(ctx, cfg, scheme, kubeCli, nsName, "curl", "curl", curlCmd)
+				gomega.Expect(err).NotTo(gomega.HaveOccurred())
+			}
+
+			// modify the curl command to generate some error metrics
+			curlCmd[len(curlCmd)-1] = "invalid input"
+			for i := 0; i < 5; i++ {
+				_, err := testutils.ExecCommandInPod(ctx, cfg, scheme, kubeCli, nsName, "curl", "curl", curlCmd)
+				gomega.Expect(err).NotTo(gomega.HaveOccurred())
+			}
+
+			// Now scrape metrics from the EPP endpoint via the curl pod
+			ginkgo.By("Scraping metrics from the EPP endpoint")
+
+			// Get Pod IP instead of Service
+			podList := &corev1.PodList{}
+			err := cli.List(ctx, podList, client.InNamespace(nsName), client.MatchingLabels{"app": inferExtName})
+			gomega.Expect(err).NotTo(gomega.HaveOccurred())
+			gomega.Expect(podList.Items).NotTo(gomega.BeEmpty())
+			podIP := podList.Items[0].Status.PodIP
+			gomega.Expect(podIP).NotTo(gomega.BeEmpty())
+
+			// Get the authorization token for reading metrics
+			token := ""
+			gomega.Eventually(func() error {
+				token, err = getMetricsReaderToken(cli)
+				if err != nil {
+					return err
+				}
+				if token == "" {
+					return errors.New("token not found")
+				}
+				return nil
+			}, existsTimeout, interval).Should(gomega.Succeed())
+
+			// Construct the metric scraping curl command using Pod IP
+			metricScrapeCmd := []string{
+				"curl",
+				"-i",
+				"--max-time",
+				strconv.Itoa((int)(curlTimeout.Seconds())),
+				"-H",
+				"Authorization: Bearer " + token,
+				fmt.Sprintf("http://%s:%d/metrics", podIP, 9090),
+			}
+
+			ginkgo.By("Verifying that all expected metrics are present.")
+			gomega.Eventually(func() error {
+				// Execute the metrics scrape command inside the curl pod
+				resp, err := testutils.ExecCommandInPod(ctx, cfg, scheme, kubeCli, nsName, "curl", "curl", metricScrapeCmd)
+				if err != nil {
+					return err
+				}
+				// Verify that we got a 200 OK responsecurl
+				if !strings.Contains(resp, "200 OK") {
+					return fmt.Errorf("did not get 200 OK: %s", resp)
+				}
+				// Check if all expected metrics are present in the metrics output
+				for _, metric := range expectedMetrics {
+					if !strings.Contains(resp, metric) {
+						return fmt.Errorf("expected metric %s not found in metrics output", metric)
+					}
+				}
+				return nil
+			}, readyTimeout, curlInterval).Should(gomega.Succeed())
+		})
 	})
 })
 
@@ -130,16 +246,38 @@ func newInferenceModel(ns string) *v1alpha2.InferenceModel {
 		Obj()
 }
 
+func getMetricsReaderToken(k8sClient client.Client) (string, error) {
+	secret := &corev1.Secret{}
+	err := k8sClient.Get(ctx, types.NamespacedName{Namespace: nsName, Name: metricsReaderSecretName}, secret)
+	if err != nil {
+		return "", err
+	}
+	return string(secret.Data["token"]), nil
+}
+
 // getCurlCommand returns the command, as a slice of strings, for curl'ing
 // the test model server at the given name, namespace, port, and model name.
-func getCurlCommand(name, ns, port, model string, timeout time.Duration, api string, promptOrMessages string) []string {
-	var body string
+func getCurlCommand(name, ns, port, model string, timeout time.Duration, api string, promptOrMessages any, streaming bool) []string {
+	body := map[string]any{
+		"model":       model,
+		"max_tokens":  100,
+		"temperature": 0,
+	}
+	body["model"] = model
 	switch api {
 	case "/completions":
-		body = fmt.Sprintf(`{"model": "%s", "prompt": "%s", "max_tokens": 100, "temperature": 0}`, model, promptOrMessages)
+		body["prompt"] = promptOrMessages
 	case "/chat/completions":
-		body = fmt.Sprintf(`{"model": "%s", "messages": %s, "max_tokens": 100, "temperature": 0}`, model, promptOrMessages)
+		body["messages"] = promptOrMessages
+	}
+	if streaming {
+		body["stream"] = true
+		body["stream_options"] = map[string]any{
+			"include_usage": true,
+		}
 	}
+	b, err := json.Marshal(body)
+	gomega.Expect(err).NotTo(gomega.HaveOccurred())
 	return []string{
 		"curl",
 		"-i",
@@ -149,6 +287,6 @@ func getCurlCommand(name, ns, port, model string, timeout time.Duration, api str
 		"-H",
 		"Content-Type: application/json",
 		"-d",
-		body,
+		string(b),
 	}
 }
diff --git a/test/integration/epp/hermetic_test.go b/test/integration/epp/hermetic_test.go
index 5f473b4..db62529 100644
--- a/test/integration/epp/hermetic_test.go
+++ b/test/integration/epp/hermetic_test.go
@@ -925,7 +925,7 @@ func BeforeSuite() func() {
 	// Adjust from defaults
 	serverRunner.PoolNamespacedName = types.NamespacedName{Name: testPoolName, Namespace: testNamespace}
 	serverRunner.Datastore = datastore.NewDatastore(context.Background(), pmf)
-	scheduler := scheduling.NewScheduler(serverRunner.Datastore)
+	scheduler := scheduling.NewScheduler()
 
 	sdConfig := &saturationdetector.Config{
 		QueueDepthThreshold:       saturationdetector.DefaultQueueDepthThreshold,
diff --git a/test/testdata/configloader_1_test.yaml b/test/testdata/configloader_1_test.yaml
index bd66b45..f1f167e 100644
--- a/test/testdata/configloader_1_test.yaml
+++ b/test/testdata/configloader_1_test.yaml
@@ -2,16 +2,16 @@ apiVersion: inference.networking.x-k8s.io/v1alpha1
 kind: EndpointPickerConfig
 plugins:
 - name: test1
-  pluginName: test-one
+  type: test-one
   parameters:
     threshold: 10
 - name: profileHandler
-  pluginName: test-profile-handler
-- pluginName: test-two
+  type: test-profile-handler
+- type: test-two
   parameters:
     hashBlockSize: 32
 - name: testPicker
-  pluginName: test-picker
+  type: test-picker
 
 schedulingProfiles:
 - name: default
diff --git a/test/testdata/inferencepool-e2e.yaml b/test/testdata/inferencepool-e2e.yaml
index 982a1d4..261b00b 100644
--- a/test/testdata/inferencepool-e2e.yaml
+++ b/test/testdata/inferencepool-e2e.yaml
@@ -119,5 +119,6 @@ subjects:
   name: default
   namespace: $E2E_NS
 roleRef:
+  apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: pod-read
diff --git a/test/testdata/metrics-rbac.yaml b/test/testdata/metrics-rbac.yaml
new file mode 100644
index 0000000..0518916
--- /dev/null
+++ b/test/testdata/metrics-rbac.yaml
@@ -0,0 +1,37 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: inference-gateway-metrics-reader
+rules:
+- nonResourceURLs:
+  - /metrics
+  verbs:
+  - get
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: inference-gateway-sa-metrics-reader
+  namespace: $E2E_NS
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: inference-gateway-sa-metrics-reader-role-binding
+subjects:
+- kind: ServiceAccount
+  name: inference-gateway-sa-metrics-reader
+  namespace: $E2E_NS
+roleRef:
+  kind: ClusterRole
+  name: inference-gateway-metrics-reader
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: inference-gateway-sa-metrics-reader-secret
+  namespace: $E2E_NS
+  annotations:
+    kubernetes.io/service-account.name: inference-gateway-sa-metrics-reader
+type: kubernetes.io/service-account-token
\ No newline at end of file
diff --git a/test/utils/handle.go b/test/utils/handle.go
new file mode 100644
index 0000000..3358ad8
--- /dev/null
+++ b/test/utils/handle.go
@@ -0,0 +1,60 @@
+/*
+Copyright 2024 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package utils
+
+import "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
+
+// testHandle is an implmentation of plugins.Handle for test purposes
+type testHandle struct {
+	plugins plugins.HandlePlugins
+}
+
+func (h *testHandle) Plugins() plugins.HandlePlugins {
+	return h.plugins
+}
+
+type testHandlePlugins struct {
+	thePlugins map[string]plugins.Plugin
+}
+
+func (h *testHandlePlugins) Plugin(name string) plugins.Plugin {
+	return h.thePlugins[name]
+}
+
+func (h *testHandlePlugins) AddPlugin(name string, plugin plugins.Plugin) {
+	h.thePlugins[name] = plugin
+}
+
+func (h *testHandlePlugins) GetAllPlugins() []plugins.Plugin {
+	result := make([]plugins.Plugin, 0)
+	for _, plugin := range h.thePlugins {
+		result = append(result, plugin)
+	}
+	return result
+}
+
+func (h *testHandlePlugins) GetAllPluginsWithNames() map[string]plugins.Plugin {
+	return h.thePlugins
+}
+
+func NewTestHandle() plugins.Handle {
+	return &testHandle{
+		plugins: &testHandlePlugins{
+			thePlugins: map[string]plugins.Plugin{},
+		},
+	}
+}
diff --git a/test/utils/utils.go b/test/utils/utils.go
index 1ec0fba..ba74069 100644
--- a/test/utils/utils.go
+++ b/test/utils/utils.go
@@ -59,6 +59,24 @@ func DeleteClusterResources(ctx context.Context, cli client.Client) error {
 	if err != nil && !apierrors.IsNotFound(err) {
 		return err
 	}
+	metricsReaderBinding := &rbacv1.ClusterRoleBinding{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "inference-gateway-sa-metrics-reader-role-binding",
+		},
+	}
+	err = cli.Delete(ctx, metricsReaderBinding, client.PropagationPolicy(metav1.DeletePropagationForeground))
+	if err != nil && !apierrors.IsNotFound(err) {
+		return err
+	}
+	metricsReaderRole := &rbacv1.ClusterRole{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "inference-gateway-metrics-reader",
+		},
+	}
+	err = cli.Delete(ctx, metricsReaderRole, client.PropagationPolicy(metav1.DeletePropagationForeground))
+	if err != nil && !apierrors.IsNotFound(err) {
+		return err
+	}
 	model := &apiextv1.CustomResourceDefinition{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: "inferencemodels.inference.networking.x-k8s.io",
@@ -106,6 +124,10 @@ func DeleteNamespacedResources(ctx context.Context, cli client.Client, ns string
 	if err != nil && !apierrors.IsNotFound(err) {
 		return err
 	}
+	err = cli.DeleteAllOf(ctx, &corev1.ServiceAccount{}, client.InNamespace(ns), client.PropagationPolicy(metav1.DeletePropagationForeground))
+	if err != nil && !apierrors.IsNotFound(err) {
+		return err
+	}
 	err = cli.DeleteAllOf(ctx, &v1alpha2.InferencePool{}, client.InNamespace(ns), client.PropagationPolicy(metav1.DeletePropagationForeground))
 	if err != nil && !apierrors.IsNotFound(err) {
 		return err