Repository: kubernetes-sigs/gateway-api-inference-extension

Issue:
EPP: Use Dedicated Service Account
Currently, the EPP manifest (`ext_proc.yaml`) creates clusterrolebindings and clusterroles resources to allow the informers to get/list/watch resources. However, roles and rolebindings resources can be used for inferencemodels, inferencepools, and endpointslices. For example:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-read
  namespace: default
rules:
- apiGroups: ["inference.networking.x-k8s.io"]
  resources: ["inferencemodels"]
  verbs: ["get", "watch", "list"]
- apiGroups: ["inference.networking.x-k8s.io"]
  resources: ["inferencepools"]
  verbs: ["get", "watch", "list"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
- apiGroups: ["discovery.k8s.io"]
  resources: ["endpointslices"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-read-binding
  namespace: default
subjects:
- kind: ServiceAccount
  name: default # We should create a service account for EPP
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pod-read
```

Additionally, why are the subjectaccessreviews and tokenreviews resources required? 

Hints from issue comments:
@kfswain: What is the suggested change here? clusterrolebinding is necessary for cross-namespace, which would be necessary for multitenancy, correct?@danehans: > clusterrolebinding is necessary for cross-namespace, which would be necessary for multitenancy, correct?

Right, but currently the EPP only needs to read namespace-local resources. ClusterRole/ClusterRoleBinding should be changed to Role/RoleBinding.

Additionally, why does EPP require `create` privileges for `subjectaccessreviews` and `tokenreviews`?

```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pod-read
...
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
```
@danehans: @kfswain @ahg-g do you know why ^ RBAC rules are required?@danehans: Metrics are authenticated (via TokenReviews) and authorized (via SubjectAccessReviews) with the kube-apiserver. For the authentication and authorization, EPP needs a ClusterRole with the following rules:
```yaml 
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
```@danehans: Keeping this issue open since the EPP deployment should use a dedicated service account instead of the default.@nirrozenbaum: once a dedicated service account is used and as long as EPP doesn't support multi tenancy (e.g., multi-namespace), I suggest to split current RBAC into namespace scoped permissions (InfPool, InfModel, Pod) and cluster scoped permission (tokenreviews, subjectaccessreviews). best practice is to follow least privilege principle. 
in current setup where EPP supports only one pool and acts in the scope of a namespace, ideally it would be good if we can get rid of the cluster wide rbac completely.

Solution patch:
diff --git a/Makefile b/Makefile
index 9e6e5a7..a2563a6 100644
--- a/Makefile
+++ b/Makefile
@@ -24,6 +24,7 @@ SHELL = /usr/bin/env bash -o pipefail
 GIT_COMMIT_SHA ?= "$(shell git rev-parse HEAD 2>/dev/null)"
 GIT_TAG ?= $(shell git describe --tags --dirty --always)
 PLATFORMS ?= linux/amd64
+PUBLISH_PLATFORMS ?= linux/amd64,linux/arm64
 DOCKER_BUILDX_CMD ?= docker buildx
 IMAGE_BUILD_CMD ?= $(DOCKER_BUILDX_CMD) build
 IMAGE_BUILD_EXTRA_OPTS ?=
@@ -73,6 +74,12 @@ SYNCER_IMAGE_BUILD_EXTRA_OPTS += -t $(SYNCER_IMAGE_EXTRA_TAG)
 BBR_IMAGE_BUILD_EXTRA_OPTS += -t $(BBR_IMAGE_EXTRA_TAG)
 endif
 
+# Allow `make MULTI=true ...` in CI to switch to multi-arch.
+ifdef MULTI
+  PLATFORMS := $(PUBLISH_PLATFORMS)
+endif
+
+
 # The name of the kind cluster to use for the "kind-load" target.
 KIND_CLUSTER ?= kind
 
@@ -204,7 +211,7 @@ image-build: ## Build the EPP image using Docker Buildx.
 
 .PHONY: image-push
 image-push: PUSH=--push ## Build the EPP image and push it to $IMAGE_REPO.
-image-push: image-build
+image-push: MULTI=true image-build
 
 .PHONY: image-load
 image-load: LOAD=--load ## Build the EPP image and load it in the local Docker registry.
diff --git a/cmd/epp/runner/runner.go b/cmd/epp/runner/runner.go
index d7613a9..76e05da 100644
--- a/cmd/epp/runner/runner.go
+++ b/cmd/epp/runner/runner.go
@@ -42,7 +42,7 @@ import (
 
 	"sigs.k8s.io/gateway-api-inference-extension/internal/runnable"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config/loader"
+	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/config/loader"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/datastore"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/metrics/collectors"
diff --git a/config/charts/inferencepool/templates/rbac.yaml b/config/charts/inferencepool/templates/rbac.yaml
index 217fd24..0c432f4 100644
--- a/config/charts/inferencepool/templates/rbac.yaml
+++ b/config/charts/inferencepool/templates/rbac.yaml
@@ -5,15 +5,6 @@ metadata:
   labels:
     {{- include "gateway-api-inference-extension.labels" . | nindent 4 }}
 rules:
-- apiGroups: ["inference.networking.x-k8s.io"]
-  resources: ["inferencemodels", "inferencepools"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: ["inference.networking.k8s.io"]
-  resources: ["inferencepools"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: [""]
-  resources: ["pods"]
-  verbs: ["get", "watch", "list"]
 - apiGroups:
   - authentication.k8s.io
   resources:
@@ -40,6 +31,35 @@ roleRef:
   kind: ClusterRole
   name: {{ include "gateway-api-inference-extension.name" . }}
 ---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: {{ include "gateway-api-inference-extension.name" . }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "gateway-api-inference-extension.labels" . | nindent 4 }}
+rules:
+- apiGroups: ["inference.networking.x-k8s.io"]
+  resources: ["inferencemodels", "inferencepools"]
+  verbs: ["get", "watch", "list"]
+- apiGroups: [""]
+  resources: ["pods"]
+  verbs: ["get", "watch", "list"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: {{ include "gateway-api-inference-extension.name" . }}
+  namespace: {{ .Release.Namespace }}
+subjects:
+- kind: ServiceAccount
+  name: {{ include "gateway-api-inference-extension.name" . }}
+  namespace: {{ .Release.Namespace }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: {{ include "gateway-api-inference-extension.name" . }}
+---
 apiVersion: v1
 kind: ServiceAccount
 metadata:
diff --git a/config/manifests/inferencepool-resources.yaml b/config/manifests/inferencepool-resources.yaml
index fa49b25..9c6d7c2 100644
--- a/config/manifests/inferencepool-resources.yaml
+++ b/config/manifests/inferencepool-resources.yaml
@@ -29,6 +29,12 @@ spec:
       appProtocol: http2
   type: ClusterIP
 ---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: vllm-llama3-8b-instruct-epp
+  namespace: default
+---
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -46,6 +52,7 @@ spec:
       labels:
         app: vllm-llama3-8b-instruct-epp
     spec:
+      serviceAccountName: vllm-llama3-8b-instruct-epp
       # Conservatively, this timeout should mirror the longest grace period of the pods within the pool
       terminationGracePeriodSeconds: 130
       containers:
@@ -174,23 +181,41 @@ data:
         weight: 1
       - pluginRef: max-score-picker
 ---
-kind: ClusterRole
+kind: Role
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: pod-read
+  namespace: default
+rules:
+- apiGroups: [ "inference.networking.x-k8s.io" ]
+  resources: [ "inferencemodels", "inferencepools" ]
+  verbs: [ "get", "watch", "list" ]
+- apiGroups: [ "inference.networking.k8s.io" ]
+  resources: [ "inferencepools" ]
+  verbs: [ "get", "watch", "list" ]
+- apiGroups: [ "" ]
+  resources: [ "pods" ]
+  verbs: [ "get", "watch", "list" ]
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: pod-read-binding
+  namespace: default
+subjects:
+- kind: ServiceAccount
+  name: vllm-llama3-8b-instruct-epp
+  namespace: default
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: pod-read
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: auth-reviewer
 rules:
-- apiGroups: ["inference.networking.k8s.io"]
-  resources: ["inferencepools"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: ["inference.networking.x-k8s.io"]
-  resources: ["inferencepools"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: ["inference.networking.x-k8s.io"]
-  resources: ["inferencemodels"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: [""]
-  resources: ["pods"]
-  verbs: ["get", "watch", "list"]
 - apiGroups:
   - authentication.k8s.io
   resources:
@@ -207,12 +232,12 @@ rules:
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: pod-read-binding
+  name: auth-reviewer-binding
 subjects:
 - kind: ServiceAccount
-  name: default
+  name: vllm-llama3-8b-instruct-epp
   namespace: default
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
-  name: pod-read
+  name: auth-reviewer
diff --git a/go.mod b/go.mod
index bfc86ca..a7d6848 100644
--- a/go.mod
+++ b/go.mod
@@ -4,7 +4,7 @@ go 1.24.0
 
 require (
 	github.com/cespare/xxhash/v2 v2.3.0
-	github.com/elastic/crd-ref-docs v0.1.0
+	github.com/elastic/crd-ref-docs v0.2.0
 	github.com/envoyproxy/go-control-plane/envoy v1.32.4
 	github.com/go-logr/logr v1.4.3
 	github.com/google/go-cmp v0.7.0
@@ -49,7 +49,6 @@ require (
 	github.com/emicklei/go-restful/v3 v3.12.0 // indirect
 	github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
 	github.com/evanphx/json-patch/v5 v5.9.11 // indirect
-	github.com/fatih/color v1.18.0 // indirect
 	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/fsnotify/fsnotify v1.7.0 // indirect
 	github.com/fxamacker/cbor/v2 v2.7.0 // indirect
@@ -58,11 +57,9 @@ require (
 	github.com/go-openapi/jsonpointer v0.21.0 // indirect
 	github.com/go-openapi/jsonreference v0.21.0 // indirect
 	github.com/go-openapi/swag v0.23.0 // indirect
-	github.com/go-playground/locales v0.14.1 // indirect
-	github.com/go-playground/universal-translator v0.18.0 // indirect
 	github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
 	github.com/gobuffalo/flect v1.0.3 // indirect
-	github.com/goccy/go-yaml v1.11.3 // indirect
+	github.com/goccy/go-yaml v1.18.0 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/google/btree v1.1.3 // indirect
 	github.com/google/cel-go v0.23.2 // indirect
@@ -76,10 +73,7 @@ require (
 	github.com/josharian/intern v1.0.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/kylelemons/godebug v1.1.0 // indirect
-	github.com/leodido/go-urn v1.2.1 // indirect
 	github.com/mailru/easyjson v0.7.7 // indirect
-	github.com/mattn/go-colorable v0.1.13 // indirect
-	github.com/mattn/go-isatty v0.0.20 // indirect
 	github.com/mitchellh/copystructure v1.2.0 // indirect
 	github.com/mitchellh/reflectwalk v1.0.2 // indirect
 	github.com/moby/spdystream v0.5.0 // indirect
@@ -106,17 +100,17 @@ require (
 	go.opentelemetry.io/proto/otlp v1.4.0 // indirect
 	go.uber.org/automaxprocs v1.6.0 // indirect
 	go.yaml.in/yaml/v2 v2.4.2 // indirect
-	golang.org/x/crypto v0.39.0 // indirect
+	golang.org/x/crypto v0.40.0 // indirect
 	golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
-	golang.org/x/mod v0.25.0 // indirect
-	golang.org/x/net v0.41.0 // indirect
+	golang.org/x/mod v0.26.0 // indirect
+	golang.org/x/net v0.42.0 // indirect
 	golang.org/x/oauth2 v0.30.0 // indirect
-	golang.org/x/sys v0.33.0 // indirect
-	golang.org/x/term v0.32.0 // indirect
-	golang.org/x/text v0.26.0 // indirect
+	golang.org/x/sys v0.34.0 // indirect
+	golang.org/x/term v0.33.0 // indirect
+	golang.org/x/text v0.27.0 // indirect
 	golang.org/x/time v0.9.0 // indirect
-	golang.org/x/tools v0.33.0 // indirect
-	golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
+	golang.org/x/tools v0.35.0 // indirect
+	golang.org/x/tools/go/expect v0.1.1-deprecated // indirect
 	gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20250528174236-200df99c418a // indirect
diff --git a/go.sum b/go.sum
index 67469b8..58fb085 100644
--- a/go.sum
+++ b/go.sum
@@ -25,8 +25,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/elastic/crd-ref-docs v0.1.0 h1:Cr5kz89QB3Iuuj7dhAfLMApCrChEGAaIBTxGk/xuRKw=
-github.com/elastic/crd-ref-docs v0.1.0/go.mod h1:X83mMBdJt05heJUYiS3T0yJ/JkCuliuhSUNav5Gjo/U=
+github.com/elastic/crd-ref-docs v0.2.0 h1:U17MyGX71j4qfKTvYxbR4qZGoA1hc2thy7kseGYmP+o=
+github.com/elastic/crd-ref-docs v0.2.0/go.mod h1:0bklkJhTG7nC6AVsdDi0wt5bGoqvzdZSzMMQkilZ6XM=
 github.com/emicklei/go-restful/v3 v3.12.0 h1:y2DdzBAURM29NFF94q6RaY4vjIH1rtwDapwQtU84iWk=
 github.com/emicklei/go-restful/v3 v3.12.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
 github.com/envoyproxy/go-control-plane/envoy v1.32.4 h1:jb83lalDRZSpPWW2Z7Mck/8kXZ5CQAFYVjQcdVIr83A=
@@ -37,8 +37,6 @@ github.com/evanphx/json-patch v0.5.2 h1:xVCHIVMUu1wtM/VkR9jVZ45N3FhZfYMMYGorLCR8
 github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ=
 github.com/evanphx/json-patch/v5 v5.9.11 h1:/8HVnzMq13/3x9TPvjG08wUGqBTmZBsCWzjTM0wiaDU=
 github.com/evanphx/json-patch/v5 v5.9.11/go.mod h1:3j+LviiESTElxA4p3EMKAB9HXj3/XEtnUf6OZxqIQTM=
-github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
-github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
 github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
 github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
 github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
@@ -58,19 +56,12 @@ github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF
 github.com/go-openapi/jsonreference v0.21.0/go.mod h1:LmZmgsrTkVg9LG4EaHeY8cBDslNPMo06cago5JNLkm4=
 github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
 github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
-github.com/go-playground/locales v0.14.0/go.mod h1:sawfccIbzZTqEDETgFXqTho0QybSa7l++s0DH+LDiLs=
-github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
-github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
-github.com/go-playground/universal-translator v0.18.0 h1:82dyy6p4OuJq4/CByFNOn/jYrnRPArHwAcmLoJZxyho=
-github.com/go-playground/universal-translator v0.18.0/go.mod h1:UvRDBj+xPUEGrFYl+lu/H90nyDXpg0fqeB/AQUGNTVA=
-github.com/go-playground/validator/v10 v10.4.1 h1:pH2c5ADXtd66mxoE0Zm9SUhxE20r7aM3F26W0hOn+GE=
-github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
 github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
 github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
 github.com/gobuffalo/flect v1.0.3 h1:xeWBM2nui+qnVvNM4S3foBhCAL2XgPU+a7FdpelbTq4=
 github.com/gobuffalo/flect v1.0.3/go.mod h1:A5msMlrHtLqh9umBSnvabjsMrCcCpAyzglnDvkbYKHs=
-github.com/goccy/go-yaml v1.11.3 h1:B3W9IdWbvrUu2OYQGwvU1nZtvMQJPBKgBUuweJjLj6I=
-github.com/goccy/go-yaml v1.11.3/go.mod h1:wKnAMd44+9JAAnGQpWVEgBzGt3YuTaQ4uXoHvE4m7WU=
+github.com/goccy/go-yaml v1.18.0 h1:8W7wMFS12Pcas7KU+VVkaiCng+kG8QiFeFwzFb+rwuw=
+github.com/goccy/go-yaml v1.18.0/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
 github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
 github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
 github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
@@ -117,15 +108,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
-github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w=
-github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
 github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
-github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
-github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
-github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
-github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
-github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
 github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
 github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
@@ -181,7 +165,6 @@ github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpE
 github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
 github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
-github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
@@ -226,20 +209,20 @@ go.yaml.in/yaml/v3 v3.0.3/go.mod h1:tBHosrYAkRZjRAOREWbDnBXUf08JOwYq++0QNwQiWzI=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
-golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
-golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
+golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
+golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
 golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8=
 golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY=
 golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
-golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
+golang.org/x/mod v0.26.0 h1:EGMPT//Ezu+ylkCijjPc+f4Aih7sZvaAr+O3EHBxvZg=
+golang.org/x/mod v0.26.0/go.mod h1:/j6NAhSk8iQ723BGAUyoAcn7SlD7s15Dp9Nd/SfeaFQ=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
-golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
+golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
+golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
 golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
 golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -250,31 +233,30 @@ golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
-golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
-golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
-golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
+golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
+golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
+golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
+golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
-golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
+golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
+golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
 golang.org/x/time v0.9.0 h1:EsRrnYcQiGH+5FfbgvV4AP7qEZstoyrHB0DzarOQ4ZY=
 golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.33.0 h1:4qz2S3zmRxbGIhDIAgjxvFutSvH5EfnsYrRBj0UI0bc=
-golang.org/x/tools v0.33.0/go.mod h1:CIJMaWEY88juyUfo7UbgPqbC8rU2OqfAV1h2Qp0oMYI=
+golang.org/x/tools v0.35.0 h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0=
+golang.org/x/tools v0.35.0/go.mod h1:NKdj5HkL/73byiZSJjqJgKn3ep7KjFkBOkR/Hps3VPw=
+golang.org/x/tools/go/expect v0.1.1-deprecated h1:jpBZDwmgPhXsKZC6WhL20P4b/wmnpsEAGHaNy0n/rJM=
+golang.org/x/tools/go/expect v0.1.1-deprecated/go.mod h1:eihoPOH+FgIqa3FpoTwguz/bVUSGBlGQU67vpBeOrBY=
+golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated h1:1h2MnaIAIXISqTFKdENegdpAgUXz6NrPEsbIeWaBRvM=
+golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated/go.mod h1:RVAQXBGNv1ib0J382/DPCRS/BPnsGebyM1Gj5VSDpG8=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 h1:+cNy6SZtPcJQH3LJVLOSmiC7MMxXNOb3PU/VUEz+EhU=
-golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=
 gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw=
 gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=
 google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a h1:SGktgSolFCo75dnHJF2yMvnns6jCmHFJ0vE4Vn2JKvQ=
diff --git a/pkg/epp/common/config/defaults.go b/pkg/epp/common/config/defaults.go
deleted file mode 100644
index a7b820e..0000000
--- a/pkg/epp/common/config/defaults.go
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
-Copyright 2025 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-// Package config holds common configuration default values used across
-// different EPP components.
-package config
-
-const (
-	// DefaultKVCacheThreshold is the default KV cache utilization (0.0 to 1.0)
-	// threshold.
-	DefaultKVCacheThreshold = 0.8
-	// DefaultQueueThresholdCritical is the default backend waiting queue size
-	// threshold.
-	DefaultQueueThresholdCritical = 5
-
-	// DefaultScorerWeight is the weight used for scorers referenced in the
-	// configuration without explicit weights.
-	DefaultScorerWeight = 1
-)
diff --git a/pkg/epp/common/config/loader/configloader.go b/pkg/epp/config/loader/configloader.go
similarity index 97%
rename from pkg/epp/common/config/loader/configloader.go
rename to pkg/epp/config/loader/configloader.go
index a71fda0..5e78a84 100644
--- a/pkg/epp/common/config/loader/configloader.go
+++ b/pkg/epp/config/loader/configloader.go
@@ -26,7 +26,6 @@ import (
 	"k8s.io/apimachinery/pkg/util/sets"
 
 	configapi "sigs.k8s.io/gateway-api-inference-extension/apix/config/v1alpha1"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
@@ -34,6 +33,12 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
 )
 
+const (
+	// DefaultScorerWeight is the weight used for scorers referenced in the
+	// configuration without explicit weights.
+	DefaultScorerWeight = 1
+)
+
 var scheme = runtime.NewScheme()
 
 func init() {
@@ -72,7 +77,7 @@ func LoadSchedulerConfig(configProfiles []configapi.SchedulingProfile, handle pl
 			referencedPlugin := handle.Plugin(plugin.PluginRef)
 			if scorer, ok := referencedPlugin.(framework.Scorer); ok {
 				// Set default weight if one wasn't set in the configuration
-				weight := config.DefaultScorerWeight
+				weight := DefaultScorerWeight
 				if plugin.Weight != nil {
 					weight = *plugin.Weight
 				}
diff --git a/pkg/epp/flowcontrol/controller/internal/processor.go b/pkg/epp/flowcontrol/controller/internal/processor.go
index 800ca64..0154dc7 100644
--- a/pkg/epp/flowcontrol/controller/internal/processor.go
+++ b/pkg/epp/flowcontrol/controller/internal/processor.go
@@ -549,6 +549,9 @@ func (sp *ShardProcessor) shutdown() {
 		for {
 			select {
 			case item := <-sp.enqueueChan:
+				if item == nil { // This is a safeguard against logic errors in the distributor.
+					continue
+				}
 				item.finalize(types.QueueOutcomeRejectedOther,
 					fmt.Errorf("%w: %w", types.ErrRejected, types.ErrFlowControllerShutdown))
 			default:
diff --git a/pkg/epp/metrics/metrics.go b/pkg/epp/metrics/metrics.go
index 4e9e95e..acb2f03 100644
--- a/pkg/epp/metrics/metrics.go
+++ b/pkg/epp/metrics/metrics.go
@@ -182,28 +182,17 @@ var (
 		},
 		[]string{},
 	)
-	SchedulerPluginProcessingLatencies = prometheus.NewHistogramVec(
-		prometheus.HistogramOpts{
-			Subsystem: InferenceExtension,
-			Name:      "scheduler_plugin_duration_seconds",
-			Help:      metricsutil.HelpMsgWithStability("Scheduler plugin processing latency distribution in seconds for each plugin type and plugin name.", compbasemetrics.ALPHA),
-			Buckets: []float64{
-				0.0001, 0.0002, 0.0005, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1,
-			},
-		},
-		[]string{"plugin_type", "plugin_name"},
-	)
 
-	RequestControlPluginProcessingLatencies = prometheus.NewHistogramVec(
+	PluginProcessingLatencies = prometheus.NewHistogramVec(
 		prometheus.HistogramOpts{
 			Subsystem: InferenceExtension,
-			Name:      "request_control_plugin_duration_seconds",
-			Help:      metricsutil.HelpMsgWithStability("RequestControl plugin processing latency distribution in seconds for each plugin type and plugin name.", compbasemetrics.ALPHA),
+			Name:      "plugin_duration_seconds",
+			Help:      metricsutil.HelpMsgWithStability("Plugin processing latency distribution in seconds for each extension point, plugin type and plugin name.", compbasemetrics.ALPHA),
 			Buckets: []float64{
 				0.0001, 0.0002, 0.0005, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1,
 			},
 		},
-		[]string{"plugin_type", "plugin_name"},
+		[]string{"extension_point", "plugin_type", "plugin_name"},
 	)
 
 	// Prefix indexer Metrics
@@ -265,9 +254,8 @@ func Register(customCollectors ...prometheus.Collector) {
 		metrics.Registry.MustRegister(inferencePoolAvgKVCache)
 		metrics.Registry.MustRegister(inferencePoolAvgQueueSize)
 		metrics.Registry.MustRegister(inferencePoolReadyPods)
-		metrics.Registry.MustRegister(SchedulerPluginProcessingLatencies)
 		metrics.Registry.MustRegister(SchedulerE2ELatency)
-		metrics.Registry.MustRegister(RequestControlPluginProcessingLatencies)
+		metrics.Registry.MustRegister(PluginProcessingLatencies)
 		metrics.Registry.MustRegister(InferenceExtensionInfo)
 		metrics.Registry.MustRegister(PrefixCacheSize)
 		metrics.Registry.MustRegister(PrefixCacheHitRatio)
@@ -292,9 +280,8 @@ func Reset() {
 	inferencePoolAvgKVCache.Reset()
 	inferencePoolAvgQueueSize.Reset()
 	inferencePoolReadyPods.Reset()
-	SchedulerPluginProcessingLatencies.Reset()
 	SchedulerE2ELatency.Reset()
-	RequestControlPluginProcessingLatencies.Reset()
+	PluginProcessingLatencies.Reset()
 	InferenceExtensionInfo.Reset()
 	PrefixCacheSize.Reset()
 	PrefixCacheHitRatio.Reset()
@@ -396,19 +383,14 @@ func RecordInferencePoolReadyPods(name string, runningPods float64) {
 	inferencePoolReadyPods.WithLabelValues(name).Set(runningPods)
 }
 
-// RecordSchedulerPluginProcessingLatency records the processing latency for a scheduler plugin.
-func RecordSchedulerPluginProcessingLatency(pluginType, pluginName string, duration time.Duration) {
-	SchedulerPluginProcessingLatencies.WithLabelValues(pluginType, pluginName).Observe(duration.Seconds())
-}
-
 // RecordSchedulerE2ELatency records the end-to-end scheduling latency.
 func RecordSchedulerE2ELatency(duration time.Duration) {
 	SchedulerE2ELatency.WithLabelValues().Observe(duration.Seconds())
 }
 
-// RecordRequestControlPluginProcessingLatency records the processing latency for a request-control plugin.
-func RecordRequestControlPluginProcessingLatency(pluginType, pluginName string, duration time.Duration) {
-	RequestControlPluginProcessingLatencies.WithLabelValues(pluginType, pluginName).Observe(duration.Seconds())
+// RecordPluginProcessingLatency records the processing latency for a plugin.
+func RecordPluginProcessingLatency(extensionPoint, pluginType, pluginName string, duration time.Duration) {
+	PluginProcessingLatencies.WithLabelValues(extensionPoint, pluginType, pluginName).Observe(duration.Seconds())
 }
 
 // RecordPrefixCacheSize records the size of the prefix indexer in megabytes.
diff --git a/pkg/epp/requestcontrol/director.go b/pkg/epp/requestcontrol/director.go
index a80215e..84bb862 100644
--- a/pkg/epp/requestcontrol/director.go
+++ b/pkg/epp/requestcontrol/director.go
@@ -328,22 +328,25 @@ func RandomWeightedDraw(logger logr.Logger, model *v1alpha2.InferenceModel, seed
 	return ""
 }
 
-func (d *Director) runPreRequestPlugins(ctx context.Context, request *schedulingtypes.LLMRequest, schedulingResult *schedulingtypes.SchedulingResult,
-	targetPort int,
-) {
+func (d *Director) runPreRequestPlugins(ctx context.Context, request *schedulingtypes.LLMRequest,
+	schedulingResult *schedulingtypes.SchedulingResult, targetPort int) {
+	loggerDebug := log.FromContext(ctx).V(logutil.DEBUG)
 	for _, plugin := range d.preRequestPlugins {
-		log.FromContext(ctx).V(logutil.DEBUG).Info("Running pre-request plugin", "plugin", plugin.TypedName().Type)
+		loggerDebug.Info("Running pre-request plugin", "plugin", plugin.TypedName())
 		before := time.Now()
 		plugin.PreRequest(ctx, request, schedulingResult, targetPort)
-		metrics.RecordRequestControlPluginProcessingLatency(PreRequestPluginType, plugin.TypedName().Type, time.Since(before))
+		metrics.RecordPluginProcessingLatency(PreRequestExtensionPoint, plugin.TypedName().Type, plugin.TypedName().Name, time.Since(before))
+		loggerDebug.Info("Completed running pre-request plugin successfully", "plugin", plugin.TypedName())
 	}
 }
 
 func (d *Director) runPostResponsePlugins(ctx context.Context, request *schedulingtypes.LLMRequest, response *Response, targetPod *backend.Pod) {
+	loggerDebug := log.FromContext(ctx).V(logutil.DEBUG)
 	for _, plugin := range d.postResponsePlugins {
-		log.FromContext(ctx).V(logutil.DEBUG).Info("Running post-response plugin", "plugin", plugin.TypedName().Type)
+		loggerDebug.Info("Running post-response plugin", "plugin", plugin.TypedName())
 		before := time.Now()
 		plugin.PostResponse(ctx, request, response, targetPod)
-		metrics.RecordRequestControlPluginProcessingLatency(PostResponsePluginType, plugin.TypedName().Type, time.Since(before))
+		metrics.RecordPluginProcessingLatency(PostResponseExtensionPoint, plugin.TypedName().Type, plugin.TypedName().Name, time.Since(before))
+		loggerDebug.Info("Completed running post-response plugin successfully", "plugin", plugin.TypedName())
 	}
 }
diff --git a/pkg/epp/requestcontrol/plugins.go b/pkg/epp/requestcontrol/plugins.go
index ba51c2a..ca823a6 100644
--- a/pkg/epp/requestcontrol/plugins.go
+++ b/pkg/epp/requestcontrol/plugins.go
@@ -25,8 +25,8 @@ import (
 )
 
 const (
-	PreRequestPluginType   = "PreRequest"
-	PostResponsePluginType = "PostResponse"
+	PreRequestExtensionPoint   = "PreRequest"
+	PostResponseExtensionPoint = "PostResponse"
 )
 
 // PreRequest is called by the director after a getting result from scheduling layer and
diff --git a/pkg/epp/saturationdetector/config.go b/pkg/epp/saturationdetector/config.go
index 54cd21c..b429d1e 100644
--- a/pkg/epp/saturationdetector/config.go
+++ b/pkg/epp/saturationdetector/config.go
@@ -21,14 +21,15 @@ import (
 
 	"sigs.k8s.io/controller-runtime/pkg/log"
 
-	commonconfig "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config"
 	envutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/env"
 )
 
 // Default configuration values
 const (
-	DefaultQueueDepthThreshold  = commonconfig.DefaultQueueThresholdCritical
-	DefaultKVCacheUtilThreshold = commonconfig.DefaultKVCacheThreshold
+	// DefaultQueueDepthThreshold is the default backend waiting queue size threshold.
+	DefaultQueueDepthThreshold = 5
+	// DefaultKVCacheUtilThreshold is the default KV cache utilization (0.0 to 1.0) threshold.
+	DefaultKVCacheUtilThreshold = 0.8
 	// DefaultMetricsStalenessThreshold defines how old metrics can be before they
 	// are considered stale.
 	// Given the pod metrics refresh interval is 50ms, a threshold slightly above
diff --git a/pkg/epp/scheduling/config/config.go b/pkg/epp/scheduling/config/config.go
deleted file mode 100644
index a96184d..0000000
--- a/pkg/epp/scheduling/config/config.go
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
-Copyright 2025 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package config
-
-import (
-	"sigs.k8s.io/controller-runtime/pkg/log"
-
-	commonconfig "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/common/config"
-	envutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/env"
-	logutil "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/util/logging"
-)
-
-// Config holds all the configuration values for the scheduler
-type Config struct {
-	KVCacheThreshold       float64
-	QueueThresholdCritical int
-	QueueingThresholdLoRA  int
-	LoraAffinityThreshold  float64
-}
-
-const (
-	// Default values for LoRA specific thresholds
-	DefaultQueueingThresholdLoRA = 128
-	DefaultLoraAffinityThreshold = 0.999
-)
-
-// LoadConfig loads configuration from environment variables
-func LoadConfig() Config {
-	// Use a default logger for initial configuration loading
-	baseLogger := log.Log.WithName("scheduling-config")
-
-	config := Config{
-		KVCacheThreshold:       envutil.GetEnvFloat("KV_CACHE_THRESHOLD", commonconfig.DefaultKVCacheThreshold, baseLogger),
-		QueueThresholdCritical: envutil.GetEnvInt("QUEUE_THRESHOLD_CRITICAL", commonconfig.DefaultQueueThresholdCritical, baseLogger),
-		QueueingThresholdLoRA:  envutil.GetEnvInt("QUEUING_THRESHOLD_LORA", DefaultQueueingThresholdLoRA, baseLogger),
-		LoraAffinityThreshold:  envutil.GetEnvFloat("LORA_AFFINITY_THRESHOLD", DefaultLoraAffinityThreshold, baseLogger),
-	}
-
-	baseLogger.V(logutil.DEFAULT).Info("Scheduler configuration loaded", "config", config)
-
-	return config
-}
-
-var Conf = LoadConfig()
diff --git a/pkg/epp/scheduling/framework/plugins.go b/pkg/epp/scheduling/framework/plugins.go
index 7e22d86..2c22843 100644
--- a/pkg/epp/scheduling/framework/plugins.go
+++ b/pkg/epp/scheduling/framework/plugins.go
@@ -24,12 +24,12 @@ import (
 )
 
 const (
-	ProfilePickerType          = "ProfilePicker"
-	FilterPluginType           = "Filter"
-	ScorerPluginType           = "Scorer"
-	PickerPluginType           = "Picker"
-	PostCyclePluginType        = "PostCycle"
-	ProcessProfilesResultsType = "ProcessProfilesResults"
+	ProfilePickerExtensionPoint          = "ProfilePicker"
+	FilterExtensionPoint                 = "Filter"
+	ScorerExtensionPoint                 = "Scorer"
+	PickerExtensionPoint                 = "Picker"
+	PostCycleExtensionPoint              = "PostCycle"
+	ProcessProfilesResultsExtensionPoint = "ProcessProfilesResults"
 )
 
 // ProfileHandler defines the extension points for handling multi SchedulerProfile instances.
@@ -57,6 +57,8 @@ type Filter interface {
 
 // Scorer defines the interface for scoring a list of pods based on context.
 // Scorers must score pods with a value within the range of [0,1] where 1 is the highest score.
+// If a scorer returns value greater than 1, it will be treated as score 1.
+// If a scorer returns value lower than 0, it will be treated as score 0.
 type Scorer interface {
 	plugins.Plugin
 	Score(ctx context.Context, cycleState *types.CycleState, request *types.LLMRequest, pods []types.Pod) map[types.Pod]float64
diff --git a/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go b/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go
index 79f2e5f..ad66c28 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/lora_affinity_filter.go
@@ -24,13 +24,13 @@ import (
 	"time"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
 const (
-	LoraAffinityFilterType = "lora-affinity-filter"
+	LoraAffinityFilterType       = "lora-affinity-filter"
+	DefaultLoraAffinityThreshold = 0.999
 )
 
 type loraAffinityFilterParameters struct {
@@ -42,7 +42,7 @@ var _ framework.Filter = &LoraAffinityFilter{}
 
 // LoraAffinityFilterFactory defines the factory function for LoraAffinityFilter.
 func LoraAffinityFilterFactory(name string, rawParameters json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	parameters := loraAffinityFilterParameters{Threshold: config.DefaultLoraAffinityThreshold}
+	parameters := loraAffinityFilterParameters{Threshold: DefaultLoraAffinityThreshold}
 	if rawParameters != nil {
 		if err := json.Unmarshal(rawParameters, &parameters); err != nil {
 			return nil, fmt.Errorf("failed to parse the parameters of the '%s' filter - %w", LoraAffinityFilterType, err)
diff --git a/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go b/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go
index e3d4de9..0e3b84c 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/low_queue_filter.go
@@ -22,13 +22,13 @@ import (
 	"fmt"
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
 )
 
 const (
-	LowQueueFilterType = "low-queue-filter"
+	LowQueueFilterType           = "low-queue-filter"
+	DefaultQueueingThresholdLoRA = 128
 )
 
 type lowQueueFilterParameters struct {
@@ -40,7 +40,7 @@ var _ framework.Filter = &LowQueueFilter{}
 
 // LowQueueFilterFactory defines the factory function for LowQueueFilter.
 func LowQueueFilterFactory(name string, rawParameters json.RawMessage, _ plugins.Handle) (plugins.Plugin, error) {
-	parameters := lowQueueFilterParameters{Threshold: config.DefaultQueueingThresholdLoRA}
+	parameters := lowQueueFilterParameters{Threshold: DefaultQueueingThresholdLoRA}
 	if rawParameters != nil {
 		if err := json.Unmarshal(rawParameters, &parameters); err != nil {
 			return nil, fmt.Errorf("failed to parse the parameters of the '%s' filter - %w", LowQueueFilterType, err)
diff --git a/pkg/epp/scheduling/framework/scheduler_profile.go b/pkg/epp/scheduling/framework/scheduler_profile.go
index 1b97bc4..b3b24d5 100644
--- a/pkg/epp/scheduling/framework/scheduler_profile.go
+++ b/pkg/epp/scheduling/framework/scheduler_profile.go
@@ -151,16 +151,16 @@ func (p *SchedulerProfile) runFilterPlugins(ctx context.Context, request *types.
 	loggerDebug.Info("Before running filter plugins", "pods", filteredPods)
 
 	for _, filter := range p.filters {
-		loggerDebug.Info("Running filter plugin", "plugin", filter.TypedName().Type)
+		loggerDebug.Info("Running filter plugin", "plugin", filter.TypedName())
 		before := time.Now()
 		filteredPods = filter.Filter(ctx, cycleState, request, filteredPods)
-		metrics.RecordSchedulerPluginProcessingLatency(FilterPluginType, filter.TypedName().Type, time.Since(before))
-		loggerDebug.Info("Filter plugin result", "plugin", filter.TypedName().Type, "pods", filteredPods)
+		metrics.RecordPluginProcessingLatency(FilterExtensionPoint, filter.TypedName().Type, filter.TypedName().Name, time.Since(before))
+		loggerDebug.Info("Completed running filter plugin successfully", "plugin", filter.TypedName(), "pods", filteredPods)
 		if len(filteredPods) == 0 {
 			break
 		}
 	}
-	loggerDebug.Info("After running filter plugins")
+	loggerDebug.Info("Completed running filter plugins successfully")
 
 	return filteredPods
 }
@@ -175,16 +175,16 @@ func (p *SchedulerProfile) runScorerPlugins(ctx context.Context, request *types.
 	}
 	// Iterate through each scorer in the chain and accumulate the weighted scores.
 	for _, scorer := range p.scorers {
-		loggerDebug.Info("Running scorer", "scorer", scorer.TypedName().Type)
+		loggerDebug.Info("Running scorer plugin", "plugin", scorer.TypedName())
 		before := time.Now()
 		scores := scorer.Score(ctx, cycleState, request, pods)
-		metrics.RecordSchedulerPluginProcessingLatency(ScorerPluginType, scorer.TypedName().Type, time.Since(before))
+		metrics.RecordPluginProcessingLatency(ScorerExtensionPoint, scorer.TypedName().Type, scorer.TypedName().Name, time.Since(before))
 		for pod, score := range scores { // weight is relative to the sum of weights
-			weightedScorePerPod[pod] += score * float64(scorer.Weight())
+			weightedScorePerPod[pod] += enforceScoreRange(score) * float64(scorer.Weight())
 		}
-		loggerDebug.Info("After running scorer", "scorer", scorer.TypedName().Type)
+		loggerDebug.Info("Completed running scorer plugin successfully", "plugin", scorer.TypedName())
 	}
-	loggerDebug.Info("After running scorer plugins")
+	loggerDebug.Info("Completed running scorer plugins successfully")
 
 	return weightedScorePerPod
 }
@@ -198,20 +198,32 @@ func (p *SchedulerProfile) runPickerPlugin(ctx context.Context, cycleState *type
 		i++
 	}
 
-	loggerDebug.Info("Before running picker plugin", "picker", p.picker.TypedName().Type, "pods-weighted-score", fmt.Sprint(weightedScorePerPod))
+	loggerDebug.Info("Running picker plugin", "plugin", p.picker.TypedName(), "pods-weighted-score", fmt.Sprint(weightedScorePerPod))
 	before := time.Now()
 	result := p.picker.Pick(ctx, cycleState, scoredPods)
-	metrics.RecordSchedulerPluginProcessingLatency(PickerPluginType, p.picker.TypedName().Type, time.Since(before))
-	loggerDebug.Info("After running picker plugin", "picker", p.picker.TypedName().Type, "result", result)
+	metrics.RecordPluginProcessingLatency(PickerExtensionPoint, p.picker.TypedName().Type, p.picker.TypedName().Name, time.Since(before))
+	loggerDebug.Info("Completed running picker plugin successfully", "plugin", p.picker.TypedName(), "result", result)
 
 	return result
 }
 
 func (p *SchedulerProfile) runPostCyclePlugins(ctx context.Context, cycleState *types.CycleState, result *types.ProfileRunResult) {
+	loggerDebug := log.FromContext(ctx).V(logutil.DEBUG)
 	for _, plugin := range p.postCyclePlugins {
-		log.FromContext(ctx).V(logutil.DEBUG).Info("Running post-cycle plugin", "plugin", plugin.TypedName().Type)
+		loggerDebug.Info("Running post-cycle plugin", "plugin", plugin.TypedName())
 		before := time.Now()
 		plugin.PostCycle(ctx, cycleState, result)
-		metrics.RecordSchedulerPluginProcessingLatency(PostCyclePluginType, plugin.TypedName().Type, time.Since(before))
+		metrics.RecordPluginProcessingLatency(PostCycleExtensionPoint, plugin.TypedName().Type, plugin.TypedName().Name, time.Since(before))
+		loggerDebug.Info("Completed running post-cycle plugin successfully", "plugin", plugin.TypedName())
+	}
+}
+
+func enforceScoreRange(score float64) float64 {
+	if score < 0 {
+		return 0
+	}
+	if score > 1 {
+		return 1
 	}
+	return score
 }
diff --git a/pkg/epp/scheduling/scheduler.go b/pkg/epp/scheduling/scheduler.go
index a0f52ef..12c1883 100644
--- a/pkg/epp/scheduling/scheduler.go
+++ b/pkg/epp/scheduling/scheduler.go
@@ -57,11 +57,11 @@ func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest, can
 	cycleState := types.NewCycleState()
 
 	for { // get the next set of profiles to run iteratively based on the request and the previous execution results
-		loggerDebug.Info("Running profile handler, Pick profiles", "plugin", s.profileHandler.TypedName().Type)
+		loggerDebug.Info("Running profile handler, Pick profiles", "plugin", s.profileHandler.TypedName())
 		before := time.Now()
 		profiles := s.profileHandler.Pick(ctx, cycleState, request, s.profiles, profileRunResults)
-		metrics.RecordSchedulerPluginProcessingLatency(framework.ProfilePickerType, s.profileHandler.TypedName().Type, time.Since(before))
-		loggerDebug.Info("After running profile handler Pick profiles", "plugin", s.profileHandler.TypedName().Type, "result", profiles)
+		metrics.RecordPluginProcessingLatency(framework.ProfilePickerExtensionPoint, s.profileHandler.TypedName().Type, s.profileHandler.TypedName().Name, time.Since(before))
+		loggerDebug.Info("Completed running profile handler Pick profiles successfully", "plugin", s.profileHandler.TypedName(), "result", profiles)
 		if len(profiles) == 0 { // profile picker didn't pick any profile to run
 			break
 		}
@@ -73,7 +73,7 @@ func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest, can
 			if err != nil {
 				loggerDebug.Info("failed to run scheduler profile", "profile", name, "error", err.Error())
 			} else {
-				loggerDebug.Info("After running scheduler profile succuessfully", "name", name)
+				loggerDebug.Info("Completed running scheduler profile succuessfully", "name", name)
 			}
 
 			profileRunResults[name] = profileRunResult // if profile failed to run, the run result is nil
@@ -84,11 +84,11 @@ func (s *Scheduler) Schedule(ctx context.Context, request *types.LLMRequest, can
 		return nil, fmt.Errorf("failed to run any scheduler profile for request %s", request.RequestId)
 	}
 
-	loggerDebug.Info("Running profile handler, ProcessResults", "plugin", s.profileHandler.TypedName().Type)
+	loggerDebug.Info("Running profile handler, ProcessResults", "plugin", s.profileHandler.TypedName())
 	before := time.Now()
 	result, err := s.profileHandler.ProcessResults(ctx, cycleState, request, profileRunResults)
-	metrics.RecordSchedulerPluginProcessingLatency(framework.ProcessProfilesResultsType, s.profileHandler.TypedName().Type, time.Since(before))
-	loggerDebug.Info("After running profile handler ProcessResults", "plugin", s.profileHandler.TypedName().Type)
+	metrics.RecordPluginProcessingLatency(framework.ProcessProfilesResultsExtensionPoint, s.profileHandler.TypedName().Type, s.profileHandler.TypedName().Name, time.Since(before))
+	loggerDebug.Info("Completed running profile handler ProcessResults successfully", "plugin", s.profileHandler.TypedName())
 
 	return result, err
 }



Test patch:
diff --git a/pkg/epp/common/config/loader/configloader_test.go b/pkg/epp/config/loader/configloader_test.go
similarity index 99%
rename from pkg/epp/common/config/loader/configloader_test.go
rename to pkg/epp/config/loader/configloader_test.go
index 6d2a333..aee2261 100644
--- a/pkg/epp/common/config/loader/configloader_test.go
+++ b/pkg/epp/config/loader/configloader_test.go
@@ -193,7 +193,7 @@ func TestLoadConfiguration(t *testing.T) {
 		{
 			name:       "successFromFile",
 			configText: "",
-			configFile: "../../../../../test/testdata/configloader_1_test.yaml",
+			configFile: "../../../../test/testdata/configloader_1_test.yaml",
 			want:       goodConfig,
 			wantErr:    false,
 		},
diff --git a/pkg/epp/metrics/metrics_test.go b/pkg/epp/metrics/metrics_test.go
index d2e209b..f16183d 100644
--- a/pkg/epp/metrics/metrics_test.go
+++ b/pkg/epp/metrics/metrics_test.go
@@ -559,11 +559,12 @@ func TestInferencePoolMetrics(t *testing.T) {
 	}
 }
 
-func TestSchedulerPluginProcessingLatencies(t *testing.T) {
+func TestPluginProcessingLatencies(t *testing.T) {
 	type pluginLatency struct {
-		pluginType string
-		pluginName string
-		duration   time.Duration
+		extensionPoint string
+		pluginType     string
+		pluginName     string
+		duration       time.Duration
 	}
 	scenarios := []struct {
 		name      string
@@ -573,24 +574,28 @@ func TestSchedulerPluginProcessingLatencies(t *testing.T) {
 			name: "multiple plugins",
 			latencies: []pluginLatency{
 				{
-					pluginType: "PostSchedule",
-					pluginName: "PluginB",
-					duration:   200 * time.Millisecond,
+					extensionPoint: "ProfilePicker",
+					pluginType:     "ProfileHandler",
+					pluginName:     "PluginB",
+					duration:       200 * time.Millisecond,
 				},
 				{
-					pluginType: "Filter",
-					pluginName: "PluginC",
-					duration:   50 * time.Millisecond,
+					extensionPoint: "Filter",
+					pluginType:     "TestFilter",
+					pluginName:     "PluginC",
+					duration:       50 * time.Millisecond,
 				},
 				{
-					pluginType: "Scorer",
-					pluginName: "PluginD",
-					duration:   10 * time.Millisecond,
+					extensionPoint: "Scorer",
+					pluginType:     "TestScorer",
+					pluginName:     "PluginD",
+					duration:       10 * time.Millisecond,
 				},
 				{
-					pluginType: "Picker",
-					pluginName: "PluginE",
-					duration:   10 * time.Microsecond,
+					extensionPoint: "Picker",
+					pluginType:     "TestPicker",
+					pluginName:     "PluginE",
+					duration:       10 * time.Microsecond,
 				},
 			},
 		},
@@ -599,10 +604,10 @@ func TestSchedulerPluginProcessingLatencies(t *testing.T) {
 	for _, scenario := range scenarios {
 		t.Run(scenario.name, func(t *testing.T) {
 			for _, latency := range scenario.latencies {
-				RecordSchedulerPluginProcessingLatency(latency.pluginType, latency.pluginName, latency.duration)
+				RecordPluginProcessingLatency(latency.extensionPoint, latency.pluginType, latency.pluginName, latency.duration)
 			}
 
-			wantPluginLatencies, err := os.Open("testdata/scheduler_plugin_processing_latencies_metric")
+			wantPluginLatencies, err := os.Open("testdata/plugin_processing_latencies_metric")
 			defer func() {
 				if err := wantPluginLatencies.Close(); err != nil {
 					t.Error(err)
@@ -611,7 +616,7 @@ func TestSchedulerPluginProcessingLatencies(t *testing.T) {
 			if err != nil {
 				t.Fatal(err)
 			}
-			if err := testutil.GatherAndCompare(metrics.Registry, wantPluginLatencies, "inference_extension_scheduler_plugin_duration_seconds"); err != nil {
+			if err := testutil.GatherAndCompare(metrics.Registry, wantPluginLatencies, "inference_extension_plugin_duration_seconds"); err != nil {
 				t.Error(err)
 			}
 		})
diff --git a/pkg/epp/metrics/testdata/plugin_processing_latencies_metric b/pkg/epp/metrics/testdata/plugin_processing_latencies_metric
new file mode 100644
index 0000000..9882090
--- /dev/null
+++ b/pkg/epp/metrics/testdata/plugin_processing_latencies_metric
@@ -0,0 +1,54 @@
+# HELP inference_extension_plugin_duration_seconds [ALPHA] Plugin processing latency distribution in seconds for each extension point, plugin type and plugin name.
+# TYPE inference_extension_plugin_duration_seconds histogram
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.0001"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.0002"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.0005"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.001"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.002"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.005"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.01"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.02"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.05"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="0.1"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler",le="+Inf"} 1
+inference_extension_plugin_duration_seconds_sum{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler"} 0.2
+inference_extension_plugin_duration_seconds_count{extension_point="ProfilePicker",plugin_name="PluginB",plugin_type="ProfileHandler"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.0001"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.0002"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.0005"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.001"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.002"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.005"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.01"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.02"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.05"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="0.1"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter",le="+Inf"} 1
+inference_extension_plugin_duration_seconds_sum{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter"} 0.05
+inference_extension_plugin_duration_seconds_count{extension_point="Filter",plugin_name="PluginC",plugin_type="TestFilter"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.0001"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.0002"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.0005"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.001"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.002"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.005"} 0
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.01"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.02"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.05"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="0.1"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer",le="+Inf"} 1
+inference_extension_plugin_duration_seconds_sum{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer"} 0.01
+inference_extension_plugin_duration_seconds_count{extension_point="Scorer",plugin_name="PluginD",plugin_type="TestScorer"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.0001"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.0002"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.0005"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.001"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.002"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.005"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.01"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.02"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.05"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="0.1"} 1
+inference_extension_plugin_duration_seconds_bucket{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker",le="+Inf"} 1
+inference_extension_plugin_duration_seconds_sum{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker"} 1e-05
+inference_extension_plugin_duration_seconds_count{extension_point="Picker",plugin_name="PluginE",plugin_type="TestPicker"} 1
diff --git a/pkg/epp/metrics/testdata/scheduler_plugin_processing_latencies_metric b/pkg/epp/metrics/testdata/scheduler_plugin_processing_latencies_metric
deleted file mode 100644
index 38ac8a0..0000000
--- a/pkg/epp/metrics/testdata/scheduler_plugin_processing_latencies_metric
+++ /dev/null
@@ -1,54 +0,0 @@
-# HELP inference_extension_scheduler_plugin_duration_seconds [ALPHA] Scheduler plugin processing latency distribution in seconds for each plugin type and plugin name.
-# TYPE inference_extension_scheduler_plugin_duration_seconds histogram
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.0001"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.0002"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.0005"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.001"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.002"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.005"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.01"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.02"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.05"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="0.1"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginB",plugin_type="PostSchedule",le="+Inf"} 1
-inference_extension_scheduler_plugin_duration_seconds_sum{plugin_name="PluginB",plugin_type="PostSchedule"} 0.2
-inference_extension_scheduler_plugin_duration_seconds_count{plugin_name="PluginB",plugin_type="PostSchedule"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.0001"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.0002"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.0005"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.001"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.002"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.005"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.01"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.02"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.05"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="0.1"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginC",plugin_type="Filter",le="+Inf"} 1
-inference_extension_scheduler_plugin_duration_seconds_sum{plugin_name="PluginC",plugin_type="Filter"} 0.05
-inference_extension_scheduler_plugin_duration_seconds_count{plugin_name="PluginC",plugin_type="Filter"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.0001"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.0002"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.0005"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.001"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.002"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.005"} 0
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.01"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.02"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.05"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="0.1"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginD",plugin_type="Scorer",le="+Inf"} 1
-inference_extension_scheduler_plugin_duration_seconds_sum{plugin_name="PluginD",plugin_type="Scorer"} 0.01
-inference_extension_scheduler_plugin_duration_seconds_count{plugin_name="PluginD",plugin_type="Scorer"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.0001"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.0002"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.0005"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.001"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.002"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.005"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.01"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.02"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.05"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="0.1"} 1
-inference_extension_scheduler_plugin_duration_seconds_bucket{plugin_name="PluginE",plugin_type="Picker",le="+Inf"} 1
-inference_extension_scheduler_plugin_duration_seconds_sum{plugin_name="PluginE",plugin_type="Picker"} 1e-05
-inference_extension_scheduler_plugin_duration_seconds_count{plugin_name="PluginE",plugin_type="Picker"} 1
diff --git a/pkg/epp/scheduling/framework/plugins/filter/filter_test.go b/pkg/epp/scheduling/framework/plugins/filter/filter_test.go
index 11f7a6c..646b81e 100644
--- a/pkg/epp/scheduling/framework/plugins/filter/filter_test.go
+++ b/pkg/epp/scheduling/framework/plugins/filter/filter_test.go
@@ -29,7 +29,6 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/plugins"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/scorer"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/types"
@@ -171,17 +170,8 @@ func TestLoRASoftAffinityDistribution(t *testing.T) {
 		tolerancePercent  = 5.0 // Allow 5% tolerance from expected distribution
 	)
 
-	// Save original config value to restore later
-	originalThreshold := config.Conf.LoraAffinityThreshold
-
 	// Set a specific test value for this test
 	testThreshold := 0.75 // 75%
-	config.Conf.LoraAffinityThreshold = testThreshold
-
-	// Ensure we restore the original threshold when test completes
-	defer func() {
-		config.Conf.LoraAffinityThreshold = originalThreshold
-	}()
 
 	// Create a test request and pods
 	req := &types.LLMRequest{
@@ -213,11 +203,11 @@ func TestLoRASoftAffinityDistribution(t *testing.T) {
 	availableCount := 0
 
 	// Use the test threshold value
-	expectedAffinityPercent := config.Conf.LoraAffinityThreshold * 100
+	expectedAffinityPercent := testThreshold * 100
 	expectedAvailabilityPercent := 100 - expectedAffinityPercent
 
 	// initialize LoraAffinityFilter
-	LoraAffinityFilter := NewLoraAffinityFilter(config.Conf.LoraAffinityThreshold)
+	LoraAffinityFilter := NewLoraAffinityFilter(testThreshold)
 
 	for range numIterations {
 		result := LoraAffinityFilter.Filter(context.Background(), types.NewCycleState(), req, pods)
@@ -247,8 +237,8 @@ func TestLoRASoftAffinityDistribution(t *testing.T) {
 	availableUpperBound := expectedAvailabilityPercent + tolerancePercent
 
 	t.Logf("Distribution results over %d iterations:", numIterations)
-	t.Logf("Expected affinity percent: %.2f%% (threshold: %.2f)", expectedAffinityPercent, config.Conf.LoraAffinityThreshold)
-	t.Logf("Expected availability percent: %.2f%% (threshold: %.2f)", expectedAvailabilityPercent, config.Conf.LoraAffinityThreshold)
+	t.Logf("Expected affinity percent: %.2f%% (threshold: %.2f)", expectedAffinityPercent, testThreshold)
+	t.Logf("Expected availability percent: %.2f%% (threshold: %.2f)", expectedAvailabilityPercent, testThreshold)
 	t.Logf("Actual affinity percent: %.2f%% (%d out of %d)", actualAffinityPercent, affinityCount, numIterations)
 	t.Logf("Actual available percent: %.2f%% (%d out of %d)", actualAvailablePercent, availableCount, numIterations)
 
@@ -268,8 +258,8 @@ func TestDecisionTreeFilterFactory(t *testing.T) {
 
 	leastKvCacheFilter := NewLeastKVCacheFilter()
 	leastQueueFilter := NewLeastQueueFilter()
-	loraAffinityFilter := NewLoraAffinityFilter(config.Conf.LoraAffinityThreshold)
-	lowQueueFilter := NewLowQueueFilter(config.Conf.QueueingThresholdLoRA)
+	loraAffinityFilter := NewLoraAffinityFilter(DefaultLoraAffinityThreshold)
+	lowQueueFilter := NewLowQueueFilter(DefaultQueueingThresholdLoRA)
 
 	kvCacheScorer := scorer.NewKVCacheUtilizationScorer()
 
diff --git a/pkg/epp/scheduling/scheduler_test.go b/pkg/epp/scheduling/scheduler_test.go
index 9e5d950..ea12375 100644
--- a/pkg/epp/scheduling/scheduler_test.go
+++ b/pkg/epp/scheduling/scheduler_test.go
@@ -26,7 +26,6 @@ import (
 
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend"
 	backendmetrics "sigs.k8s.io/gateway-api-inference-extension/pkg/epp/backend/metrics" // Import config for thresholds
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/filter"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
@@ -36,12 +35,12 @@ import (
 
 // Tests the default scheduler configuration and expected behavior.
 func TestSchedule(t *testing.T) {
-	loraAffinityFilter := filter.NewLoraAffinityFilter(config.Conf.LoraAffinityThreshold)
+	loraAffinityFilter := filter.NewLoraAffinityFilter(filter.DefaultLoraAffinityThreshold)
 	leastQueueFilter := filter.NewLeastQueueFilter()
 	leastKvCacheFilter := filter.NewLeastKVCacheFilter()
 
 	lowLatencyFilter := &filter.DecisionTreeFilter{
-		Current: filter.NewLowQueueFilter(config.Conf.QueueingThresholdLoRA),
+		Current: filter.NewLowQueueFilter(filter.DefaultQueueingThresholdLoRA),
 		NextOnSuccess: &filter.DecisionTreeFilter{
 			Current: loraAffinityFilter,
 			NextOnSuccessOrFailure: &filter.DecisionTreeFilter{
diff --git a/test/e2e/epp/e2e_suite_test.go b/test/e2e/epp/e2e_suite_test.go
index e4ca604..0e8ee12 100644
--- a/test/e2e/epp/e2e_suite_test.go
+++ b/test/e2e/epp/e2e_suite_test.go
@@ -392,14 +392,29 @@ func createInferExt(k8sClient client.Client, filePath string) {
 	ginkgo.By("Creating inference extension resources from manifest: " + filePath)
 	createObjsFromYaml(k8sClient, outManifests)
 
+	// Wait for the serviceaccount to exist.
+	testutils.EventuallyExists(ctx, func() error {
+		return k8sClient.Get(ctx, types.NamespacedName{Namespace: nsName, Name: inferExtName}, &corev1.ServiceAccount{})
+	}, existsTimeout, interval)
+
+	// Wait for the role to exist.
+	testutils.EventuallyExists(ctx, func() error {
+		return k8sClient.Get(ctx, types.NamespacedName{Namespace: nsName, Name: "pod-read"}, &rbacv1.Role{})
+	}, existsTimeout, interval)
+
+	// Wait for the rolebinding to exist.
+	testutils.EventuallyExists(ctx, func() error {
+		return k8sClient.Get(ctx, types.NamespacedName{Namespace: nsName, Name: "pod-read-binding"}, &rbacv1.RoleBinding{})
+	}, existsTimeout, interval)
+
 	// Wait for the clusterrole to exist.
 	testutils.EventuallyExists(ctx, func() error {
-		return k8sClient.Get(ctx, types.NamespacedName{Name: "pod-read"}, &rbacv1.ClusterRole{})
+		return k8sClient.Get(ctx, types.NamespacedName{Name: "auth-reviewer"}, &rbacv1.ClusterRole{})
 	}, existsTimeout, interval)
 
 	// Wait for the clusterrolebinding to exist.
 	testutils.EventuallyExists(ctx, func() error {
-		return k8sClient.Get(ctx, types.NamespacedName{Name: "pod-read-binding"}, &rbacv1.ClusterRoleBinding{})
+		return k8sClient.Get(ctx, types.NamespacedName{Name: "auth-reviewer-binding"}, &rbacv1.ClusterRoleBinding{})
 	}, existsTimeout, interval)
 
 	// Wait for the deployment to exist.
diff --git a/test/integration/epp/hermetic_test.go b/test/integration/epp/hermetic_test.go
index e788930..ad70ce7 100644
--- a/test/integration/epp/hermetic_test.go
+++ b/test/integration/epp/hermetic_test.go
@@ -66,7 +66,6 @@ import (
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/requestcontrol"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/saturationdetector"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling"
-	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/config"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/filter"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
@@ -836,6 +835,71 @@ func TestFullDuplexStreamed_KubeInferenceModelRequest(t *testing.T) {
 				},
 			},
 		},
+		{
+			name: "no backend pods are available",
+			requests: []*extProcPb.ProcessingRequest{
+				{
+					Request: &extProcPb.ProcessingRequest_RequestHeaders{
+						RequestHeaders: &extProcPb.HttpHeaders{
+							Headers: &configPb.HeaderMap{
+								Headers: []*configPb.HeaderValue{
+									{
+										Key:      "content-type",
+										RawValue: []byte("text/event-stream"),
+									},
+									{
+										Key:      "status",
+										RawValue: []byte("200"),
+									},
+								},
+							},
+							EndOfStream: true,
+						},
+					},
+				},
+			},
+			pods:        nil,
+			wantMetrics: map[string]string{},
+			wantErr:     true,
+			wantResponses: []*extProcPb.ProcessingResponse{
+				{
+					Response: &extProcPb.ProcessingResponse_ImmediateResponse{
+						ImmediateResponse: &extProcPb.ImmediateResponse{
+							Status: &envoyTypePb.HttpStatus{
+								Code: envoyTypePb.StatusCode_InternalServerError,
+							},
+							Body: []byte("inference gateway: Internal - no pods available in datastore"),
+						},
+					},
+				},
+			},
+		},
+		{
+			name: "request don't contains invalid payload, model not exist",
+			requests: []*extProcPb.ProcessingRequest{
+				{
+					Request: &extProcPb.ProcessingRequest_RequestBody{
+						RequestBody: &extProcPb.HttpBody{
+							Body:        []byte(`{"hello":"world"}`),
+							EndOfStream: true},
+					},
+				},
+			},
+			wantErr:     true,
+			wantMetrics: map[string]string{},
+			wantResponses: []*extProcPb.ProcessingResponse{
+				{
+					Response: &extProcPb.ProcessingResponse_ImmediateResponse{
+						ImmediateResponse: &extProcPb.ImmediateResponse{
+							Status: &envoyTypePb.HttpStatus{
+								Code: envoyTypePb.StatusCode_BadRequest,
+							},
+							Body: []byte("inference gateway: BadRequest - model not found in request body"),
+						},
+					},
+				},
+			},
+		},
 	}
 
 	for _, test := range tests {
@@ -1027,12 +1091,12 @@ func BeforeSuite() func() {
 	serverRunner.PoolNamespacedName = types.NamespacedName{Name: testPoolName, Namespace: testNamespace}
 	serverRunner.Datastore = datastore.NewDatastore(context.Background(), pmf)
 
-	loraAffinityFilter := filter.NewLoraAffinityFilter(config.Conf.LoraAffinityThreshold)
+	loraAffinityFilter := filter.NewLoraAffinityFilter(filter.DefaultLoraAffinityThreshold)
 	leastQueueFilter := filter.NewLeastQueueFilter()
 	leastKvCacheFilter := filter.NewLeastKVCacheFilter()
 
 	lowLatencyFilter := &filter.DecisionTreeFilter{
-		Current: filter.NewLowQueueFilter(config.Conf.QueueingThresholdLoRA),
+		Current: filter.NewLowQueueFilter(filter.DefaultQueueingThresholdLoRA),
 		NextOnSuccess: &filter.DecisionTreeFilter{
 			Current: loraAffinityFilter,
 			NextOnSuccessOrFailure: &filter.DecisionTreeFilter{
diff --git a/test/testdata/inferencepool-e2e.yaml b/test/testdata/inferencepool-e2e.yaml
index 3cd63f3..dd60aa0 100644
--- a/test/testdata/inferencepool-e2e.yaml
+++ b/test/testdata/inferencepool-e2e.yaml
@@ -26,6 +26,12 @@ spec:
       appProtocol: http2
   type: ClusterIP
 ---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: vllm-llama3-8b-instruct-epp
+  namespace: $E2E_NS
+---
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -43,6 +49,7 @@ spec:
       labels:
         app: vllm-llama3-8b-instruct-epp
     spec:
+      serviceAccountName: vllm-llama3-8b-instruct-epp
       # Conservatively, this timeout should mirror the longest grace period of the pods within the pool
       terminationGracePeriodSeconds: 130
       containers:
@@ -171,23 +178,41 @@ data:
         weight: 1
       - pluginRef: max-score-picker
 ---
-kind: ClusterRole
+kind: Role
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: pod-read
+  namespace: $E2E_NS
+rules:
+- apiGroups: [ "inference.networking.x-k8s.io" ]
+  resources: [ "inferencemodels", "inferencepools" ]
+  verbs: [ "get", "watch", "list" ]
+- apiGroups: [ "inference.networking.k8s.io" ]
+  resources: [ "inferencepools" ]
+  verbs: [ "get", "watch", "list" ]
+- apiGroups: [ "" ]
+  resources: [ "pods" ]
+  verbs: [ "get", "watch", "list" ]
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: pod-read-binding
+  namespace: $E2E_NS
+subjects:
+- kind: ServiceAccount
+  name: vllm-llama3-8b-instruct-epp
+  namespace: $E2E_NS
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: pod-read
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: auth-reviewer
 rules:
-- apiGroups: ["inference.networking.x-k8s.io"]
-  resources: ["inferencepools"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: ["inference.networking.x-k8s.io"]
-  resources: ["inferencemodels"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: ["inference.networking.k8s.io"]
-  resources: ["inferencepools"]
-  verbs: ["get", "watch", "list"]
-- apiGroups: [""]
-  resources: ["pods"]
-  verbs: ["get", "watch", "list"]
 - apiGroups:
   - authentication.k8s.io
   resources:
@@ -204,12 +229,12 @@ rules:
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: pod-read-binding
+  name: auth-reviewer-binding
 subjects:
 - kind: ServiceAccount
-  name: default
+  name: vllm-llama3-8b-instruct-epp
   namespace: $E2E_NS
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
-  name: pod-read
+  name: auth-reviewer
diff --git a/test/utils/utils.go b/test/utils/utils.go
index 71293f0..5b26c9a 100644
--- a/test/utils/utils.go
+++ b/test/utils/utils.go
@@ -45,7 +45,7 @@ import (
 func DeleteClusterResources(ctx context.Context, cli client.Client) error {
 	binding := &rbacv1.ClusterRoleBinding{
 		ObjectMeta: metav1.ObjectMeta{
-			Name: "pod-read-binding",
+			Name: "auth-reviewer-binding",
 		},
 	}
 	err := cli.Delete(ctx, binding, client.PropagationPolicy(metav1.DeletePropagationForeground))
@@ -54,7 +54,7 @@ func DeleteClusterResources(ctx context.Context, cli client.Client) error {
 	}
 	role := &rbacv1.ClusterRole{
 		ObjectMeta: metav1.ObjectMeta{
-			Name: "pod-read",
+			Name: "auth-reviewer",
 		},
 	}
 	err = cli.Delete(ctx, role, client.PropagationPolicy(metav1.DeletePropagationForeground))