Repository: kserve/kserve

Issue:
Introduce Priority for serving runtimes
/kind feature

**Problem**
- Currently, In auto selection mode, kserve serving runtime does not have a stable way to select a serving runtime if two are more serving runtimes support the same model format. 
- Currently, Kserve [selects the first serving runtime](https://github.com/kserve/kserve/blob/f2f0717a9d6341b6ec9b939bdd324b2c8c507551/pkg/controller/v1beta1/inferenceservice/components/predictor.go#L152) from a list of serving runtimes [sorted  by their respective creation timestamp descending and name ascending order](https://github.com/kserve/kserve/blob/f2f0717a9d6341b6ec9b939bdd324b2c8c507551/pkg/apis/serving/v1beta1/predictor_model.go#L162). So the users have little to no control over the runtime selection in auto select mode.


**Describe the solution you'd like**
[A clear and concise description of what you want to happen.]
- Introduce a new optional field called `Priority` in the model formats section. Through this field we can specify the priority of the serving runtimes for a sepcifice model format.
- The higher the priority value, the higher the precedence
- The priority is valid only if auto select is enabled otherwise the priority is not considered.
- If more than one serving runtime supports the model format and does not provided the priority, then the old way is followed for selecting the runtime.
-  The serving runtime with priority takes precedence over the serving runtime with priority not specified.
- The same model formats with same model version cannot have the same priority.

**Example Yaml**
```yaml
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
  name: kserve-mlserver
spec:
  annotations:
    prometheus.kserve.io/path: /metrics
    prometheus.kserve.io/port: '8080'
  containers:
    - env:
        - name: MLSERVER_MODEL_IMPLEMENTATION
          value: '{{.Labels.modelClass}}'
        - name: MLSERVER_HTTP_PORT
          value: '8080'
        - name: MLSERVER_GRPC_PORT
          value: '9000'
        - name: MODELS_DIR
          value: /mnt/models
      image: docker.io/seldonio/mlserver:1.3.2
      name: kserve-container
      resources:
        limits:
          cpu: '1'
          memory: 2Gi
        requests:
          cpu: '1'
          memory: 2Gi
  protocolVersions:
    - v2
  supportedModelFormats:
    - autoSelect: true
      name: sklearn
      priority: 2
      version: '0'
    - autoSelect: true
      name: xgboost
      priority: 2
      version: '1'
    - autoSelect: true
      name: lightgbm
      priority: 2
      version: '3'
    - autoSelect: true
      name: mlflow
      priority: 1
      version: '1'

```


**Anything else you would like to add:**
[Miscellaneous information that will assist in solving the issue.]



Discussion:

Resolving patch:
diff --git a/python/kserve/kserve/models/v1alpha1_supported_model_format.py b/python/kserve/kserve/models/v1alpha1_supported_model_format.py
--- a/python/kserve/kserve/models/v1alpha1_supported_model_format.py
+++ b/python/kserve/kserve/models/v1alpha1_supported_model_format.py
@@ -49,16 +49,18 @@ class V1alpha1SupportedModelFormat(object):
     openapi_types = {
         'auto_select': 'bool',
         'name': 'str',
+        'priority': 'int',
         'version': 'str'
     }
 
     attribute_map = {
         'auto_select': 'autoSelect',
         'name': 'name',
+        'priority': 'priority',
         'version': 'version'
     }
 
-    def __init__(self, auto_select=None, name='', version=None, local_vars_configuration=None):  # noqa: E501
+    def __init__(self, auto_select=None, name='', priority=None, version=None, local_vars_configuration=None):  # noqa: E501
         """V1alpha1SupportedModelFormat - a model defined in OpenAPI"""  # noqa: E501
         if local_vars_configuration is None:
             local_vars_configuration = Configuration()
@@ -66,6 +68,7 @@ def __init__(self, auto_select=None, name='', version=None, local_vars_configura
 
         self._auto_select = None
         self._name = None
+        self._priority = None
         self._version = None
         self.discriminator = None
 
@@ -73,6 +76,8 @@ def __init__(self, auto_select=None, name='', version=None, local_vars_configura
             self.auto_select = auto_select
         if name is not None:
             self.name = name
+        if priority is not None:
+            self.priority = priority
         if version is not None:
             self.version = version
 
@@ -122,6 +127,29 @@ def name(self, name):
 
         self._name = name
 
+    @property
+    def priority(self):
+        """Gets the priority of this V1alpha1SupportedModelFormat.  # noqa: E501
+
+        Priority of this serving runtime for auto selection. This is used to select the serving runtime if more than one serving runtime supports the same model format. The value should be greater than zero.  The higher the value, the higher the priority. Priority is not considered if AutoSelect is either false or not specified. Priority can be overridden by specifying the runtime in the InferenceService.  # noqa: E501
+
+        :return: The priority of this V1alpha1SupportedModelFormat.  # noqa: E501
+        :rtype: int
+        """
+        return self._priority
+
+    @priority.setter
+    def priority(self, priority):
+        """Sets the priority of this V1alpha1SupportedModelFormat.
+
+        Priority of this serving runtime for auto selection. This is used to select the serving runtime if more than one serving runtime supports the same model format. The value should be greater than zero.  The higher the value, the higher the priority. Priority is not considered if AutoSelect is either false or not specified. Priority can be overridden by specifying the runtime in the InferenceService.  # noqa: E501
+
+        :param priority: The priority of this V1alpha1SupportedModelFormat.  # noqa: E501
+        :type: int
+        """
+
+        self._priority = priority
+
     @property
     def version(self):
         """Gets the version of this V1alpha1SupportedModelFormat.  # noqa: E501


Test patch:
diff --git a/cmd/manager/main_test.go b/cmd/manager/main_test.go
--- a/cmd/manager/main_test.go
+++ b/cmd/manager/main_test.go
@@ -1,3 +1,19 @@
+/*
+Copyright 2023 The KServe Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
 package main
 
 import (
diff --git a/cmd/router/main_test.go b/cmd/router/main_test.go
--- a/cmd/router/main_test.go
+++ b/cmd/router/main_test.go
@@ -1,3 +1,19 @@
+/*
+Copyright 2023 The KServe Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
 package main
 
 import (
diff --git a/pkg/apis/serving/v1alpha1/servingruntime_types_test.go b/pkg/apis/serving/v1alpha1/servingruntime_types_test.go
--- a/pkg/apis/serving/v1alpha1/servingruntime_types_test.go
+++ b/pkg/apis/serving/v1alpha1/servingruntime_types_test.go
@@ -20,6 +20,7 @@ import (
 	"fmt"
 	"github.com/golang/protobuf/proto"
 	"github.com/kserve/kserve/pkg/constants"
+	"github.com/onsi/gomega"
 	"testing"
 
 	v1 "k8s.io/api/core/v1"
@@ -451,3 +452,194 @@ func TestServingRuntimeSpec_IsProtocolVersionSupported(t *testing.T) {
 		})
 	}
 }
+
+func TestServingRuntimeSpec_GetPriority(t *testing.T) {
+	endpoint := "endpoint"
+	version := "1.0"
+
+	scenarios := map[string]struct {
+		spec            ServingRuntimeSpec
+		modelFormatName string
+		expected        gomega.OmegaMatcher
+	}{
+		"Priority is provided": {
+			spec: ServingRuntimeSpec{
+				GrpcDataEndpoint: &endpoint,
+				ServingRuntimePodSpec: ServingRuntimePodSpec{
+					Containers: []v1.Container{
+						{
+							Args:    []string{"arg1", "arg2"},
+							Command: []string{"command", "command2"},
+							Env: []v1.EnvVar{
+								{Name: "name", Value: "value"},
+								{
+									Name: "fromSecret",
+									ValueFrom: &v1.EnvVarSource{
+										SecretKeyRef: &v1.SecretKeySelector{Key: "mykey"},
+									},
+								},
+							},
+							Image:           "image",
+							Name:            "name",
+							ImagePullPolicy: "IfNotPresent",
+							Resources: v1.ResourceRequirements{
+								Limits: v1.ResourceList{
+									v1.ResourceMemory: resource.MustParse("200Mi"),
+								},
+							},
+						},
+					},
+				},
+				SupportedModelFormats: []SupportedModelFormat{
+					{
+						Name:     "sklearn",
+						Version:  &version,
+						Priority: proto.Int32(1),
+					},
+					{
+						Name:     "pytorch",
+						Version:  &version,
+						Priority: proto.Int32(2),
+					},
+				},
+			},
+			modelFormatName: "pytorch",
+			expected:        gomega.Equal(proto.Int32(2)),
+		},
+		"Priority is not provided": {
+			spec: ServingRuntimeSpec{
+				GrpcDataEndpoint: &endpoint,
+				ServingRuntimePodSpec: ServingRuntimePodSpec{
+					Containers: []v1.Container{
+						{
+							Args:    []string{"arg1", "arg2"},
+							Command: []string{"command", "command2"},
+							Env: []v1.EnvVar{
+								{Name: "name", Value: "value"},
+								{
+									Name: "fromSecret",
+									ValueFrom: &v1.EnvVarSource{
+										SecretKeyRef: &v1.SecretKeySelector{Key: "mykey"},
+									},
+								},
+							},
+							Image:           "image",
+							Name:            "name",
+							ImagePullPolicy: "IfNotPresent",
+							Resources: v1.ResourceRequirements{
+								Limits: v1.ResourceList{
+									v1.ResourceMemory: resource.MustParse("200Mi"),
+								},
+							},
+						},
+					},
+				},
+				SupportedModelFormats: []SupportedModelFormat{
+					{
+						Name:    "sklearn",
+						Version: &version,
+					},
+					{
+						Name:     "pytorch",
+						Version:  &version,
+						Priority: proto.Int32(2),
+					},
+				},
+			},
+			modelFormatName: "sklearn",
+			expected:        gomega.BeNil(),
+		},
+		"If provided model format does not exist": {
+			spec: ServingRuntimeSpec{
+				GrpcDataEndpoint: &endpoint,
+				ServingRuntimePodSpec: ServingRuntimePodSpec{
+					Containers: []v1.Container{
+						{
+							Args:    []string{"arg1", "arg2"},
+							Command: []string{"command", "command2"},
+							Env: []v1.EnvVar{
+								{Name: "name", Value: "value"},
+								{
+									Name: "fromSecret",
+									ValueFrom: &v1.EnvVarSource{
+										SecretKeyRef: &v1.SecretKeySelector{Key: "mykey"},
+									},
+								},
+							},
+							Image:           "image",
+							Name:            "name",
+							ImagePullPolicy: "IfNotPresent",
+							Resources: v1.ResourceRequirements{
+								Limits: v1.ResourceList{
+									v1.ResourceMemory: resource.MustParse("200Mi"),
+								},
+							},
+						},
+					},
+				},
+				SupportedModelFormats: []SupportedModelFormat{
+					{
+						Name:     "sklearn",
+						Version:  &version,
+						Priority: proto.Int32(1),
+					},
+					{
+						Name:     "pytorch",
+						Version:  &version,
+						Priority: proto.Int32(2),
+					},
+				},
+			},
+			modelFormatName: "lightgbm",
+			expected:        gomega.BeNil(),
+		},
+	}
+	for name, scenario := range scenarios {
+		t.Run(name, func(t *testing.T) {
+			g := gomega.NewGomegaWithT(t)
+			res := scenario.spec.GetPriority(scenario.modelFormatName)
+			g.Expect(res).To(scenario.expected)
+		})
+	}
+}
+
+func TestSupportedModelFormat_IsAutoSelectEnabled(t *testing.T) {
+	version := "1.0"
+
+	scenarios := map[string]struct {
+		modelFormat SupportedModelFormat
+		res         bool
+	}{
+		"default behaviour": {
+			modelFormat: SupportedModelFormat{
+				Name:    "name",
+				Version: &version,
+			},
+			res: false,
+		},
+		"autoselect is set to true": {
+			modelFormat: SupportedModelFormat{
+				Name:       "name",
+				Version:    &version,
+				AutoSelect: proto.Bool(true),
+			},
+			res: true,
+		},
+		"autoselect is set to false": {
+			modelFormat: SupportedModelFormat{
+				Name:       "name",
+				Version:    &version,
+				AutoSelect: proto.Bool(false),
+			},
+			res: false,
+		},
+	}
+	for name, scenario := range scenarios {
+		t.Run(name, func(t *testing.T) {
+			res := scenario.modelFormat.IsAutoSelectEnabled()
+			if res != scenario.res {
+				fmt.Println(fmt.Errorf("Expected %t, got %t", scenario.res, res))
+			}
+		})
+	}
+}
diff --git a/pkg/apis/serving/v1beta1/predictor_model_test.go b/pkg/apis/serving/v1beta1/predictor_model_test.go
--- a/pkg/apis/serving/v1beta1/predictor_model_test.go
+++ b/pkg/apis/serving/v1beta1/predictor_model_test.go
@@ -37,9 +37,12 @@ func TestGetSupportingRuntimes(t *testing.T) {
 	tfRuntime := "tf-runtime"
 	sklearnRuntime := "sklearn-runtime"
 	pmmlRuntime := "pmml-runtime"
+	mlserverRuntimeMMS := "mlserver-runtime-mms"
 	mlserverRuntime := "mlserver-runtime"
 	xgboostRuntime := "xgboost-runtime"
 	clusterServingRuntimePrefix := "cluster-"
+	tritonRuntime := "triton-runtime"
+	testRuntime := "test-runtime"
 
 	protocolV2 := constants.ProtocolV2
 	protocolV1 := constants.ProtocolV1
@@ -51,9 +54,10 @@ func TestGetSupportingRuntimes(t *testing.T) {
 					Name:       "tensorflow",
 					Version:    proto.String("1"),
 					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(1),
 				},
 			},
-			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV1},
+			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV1, constants.ProtocolV2},
 			ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
 				Containers: []v1.Container{
 					{
@@ -70,9 +74,10 @@ func TestGetSupportingRuntimes(t *testing.T) {
 					Name:       "sklearn",
 					Version:    proto.String("0"),
 					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(1),
 				},
 			},
-			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV1},
+			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV1, constants.ProtocolV2},
 			ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
 				Containers: []v1.Container{
 					{
@@ -86,11 +91,12 @@ func TestGetSupportingRuntimes(t *testing.T) {
 		pmmlRuntime: {
 			SupportedModelFormats: []v1alpha1.SupportedModelFormat{
 				{
-					Name:    "pmml",
-					Version: proto.String("4"),
+					Name:     "pmml",
+					Version:  proto.String("4"),
+					Priority: proto.Int32(1),
 				},
 			},
-			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV1},
+			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV1, constants.ProtocolV2},
 			ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
 				Containers: []v1.Container{
 					{
@@ -101,22 +107,25 @@ func TestGetSupportingRuntimes(t *testing.T) {
 			},
 			Disabled: proto.Bool(true),
 		},
-		mlserverRuntime: {
+		mlserverRuntimeMMS: {
 			SupportedModelFormats: []v1alpha1.SupportedModelFormat{
 				{
 					Name:       "sklearn",
 					Version:    proto.String("0"),
 					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(2),
 				},
 				{
 					Name:       "xgboost",
 					Version:    proto.String("1"),
 					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(2),
 				},
 				{
 					Name:       "lightgbm",
 					Version:    proto.String("3"),
 					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(2),
 				},
 			},
 			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV2},
@@ -132,12 +141,40 @@ func TestGetSupportingRuntimes(t *testing.T) {
 			Disabled:                         proto.Bool(false),
 			MultiModel:                       proto.Bool(true),
 		},
+		mlserverRuntime: {
+			SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+				{
+					Name:       "sklearn",
+					Version:    proto.String("0"),
+					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(2),
+				},
+				{
+					Name:       "lightgbm",
+					Version:    proto.String("3"),
+					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(2),
+				},
+			},
+			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV2},
+			ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+				Containers: []v1.Container{
+					{
+						Name:  "kserve-container",
+						Image: mlserverRuntime + "-image:latest",
+					},
+				},
+			},
+			Disabled:   proto.Bool(false),
+			MultiModel: proto.Bool(false),
+		},
 		xgboostRuntime: {
 			SupportedModelFormats: []v1alpha1.SupportedModelFormat{
 				{
 					Name:       "xgboost",
 					Version:    proto.String("0"),
 					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(1),
 				},
 			},
 			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV2},
@@ -151,6 +188,57 @@ func TestGetSupportingRuntimes(t *testing.T) {
 			},
 			Disabled: proto.Bool(false),
 		},
+		tritonRuntime: {
+			SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+				{
+					Name:       "sklearn",
+					Version:    proto.String("0"),
+					AutoSelect: proto.Bool(true),
+				},
+				{
+					Name:       "triton",
+					Version:    proto.String("1"),
+					AutoSelect: proto.Bool(true),
+					Priority:   proto.Int32(1),
+				},
+				{
+					Name:       "lightgbm",
+					Version:    proto.String("3"),
+					AutoSelect: proto.Bool(true),
+				},
+			},
+			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV2},
+			ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+				Containers: []v1.Container{
+					{
+						Name:  "kserve-container",
+						Image: mlserverRuntime + "-image:latest",
+					},
+				},
+			},
+			Disabled:   proto.Bool(false),
+			MultiModel: proto.Bool(false),
+		},
+		testRuntime: {
+			SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+				{
+					Name:       "sklearn",
+					Version:    proto.String("0"),
+					AutoSelect: proto.Bool(true),
+				},
+			},
+			ProtocolVersions: []constants.InferenceServiceProtocol{constants.ProtocolV2},
+			ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+				Containers: []v1.Container{
+					{
+						Name:  "kserve-container",
+						Image: mlserverRuntime + "-image:latest",
+					},
+				},
+			},
+			Disabled:   proto.Bool(false),
+			MultiModel: proto.Bool(false),
+		},
 	}
 
 	runtimes := &v1alpha1.ServingRuntimeList{
@@ -176,6 +264,27 @@ func TestGetSupportingRuntimes(t *testing.T) {
 				},
 				Spec: servingRuntimeSpecs[pmmlRuntime],
 			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      mlserverRuntime,
+					Namespace: namespace,
+				},
+				Spec: servingRuntimeSpecs[mlserverRuntime],
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      tritonRuntime,
+					Namespace: namespace,
+				},
+				Spec: servingRuntimeSpecs[tritonRuntime],
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      testRuntime,
+					Namespace: namespace,
+				},
+				Spec: servingRuntimeSpecs[testRuntime],
+			},
 		},
 	}
 
@@ -183,9 +292,9 @@ func TestGetSupportingRuntimes(t *testing.T) {
 		Items: []v1alpha1.ClusterServingRuntime{
 			{
 				ObjectMeta: metav1.ObjectMeta{
-					Name: clusterServingRuntimePrefix + mlserverRuntime,
+					Name: clusterServingRuntimePrefix + mlserverRuntimeMMS,
 				},
-				Spec: servingRuntimeSpecs[mlserverRuntime],
+				Spec: servingRuntimeSpecs[mlserverRuntimeMMS],
 			},
 			{
 				ObjectMeta: metav1.ObjectMeta{
@@ -255,7 +364,7 @@ func TestGetSupportingRuntimes(t *testing.T) {
 				},
 			},
 			isMMS:    true,
-			expected: []v1alpha1.SupportedRuntime{{Name: clusterServingRuntimePrefix + mlserverRuntime, Spec: servingRuntimeSpecs[mlserverRuntime]}},
+			expected: []v1alpha1.SupportedRuntime{{Name: clusterServingRuntimePrefix + mlserverRuntimeMMS, Spec: servingRuntimeSpecs[mlserverRuntimeMMS]}},
 		},
 		"SMSRuntimeModelFormatSpecified": {
 			spec: &ModelSpec{
@@ -295,6 +404,24 @@ func TestGetSupportingRuntimes(t *testing.T) {
 			isMMS:    false,
 			expected: []v1alpha1.SupportedRuntime{},
 		},
+		"MultipleRuntimeSupportsModelFormatSpecified": {
+			spec: &ModelSpec{
+				ModelFormat: ModelFormat{
+					Name: "sklearn",
+				},
+				PredictorExtensionSpec: PredictorExtensionSpec{
+					ProtocolVersion: &protocolV2,
+					StorageURI:      &storageUri,
+				},
+			},
+			isMMS: false,
+			expected: []v1alpha1.SupportedRuntime{
+				{Name: mlserverRuntime, Spec: servingRuntimeSpecs[mlserverRuntime]},
+				{Name: sklearnRuntime, Spec: servingRuntimeSpecs[sklearnRuntime]},
+				{Name: testRuntime, Spec: servingRuntimeSpecs[testRuntime]},
+				{Name: tritonRuntime, Spec: servingRuntimeSpecs[tritonRuntime]},
+			},
+		},
 	}
 
 	s := runtime.NewScheme()
diff --git a/pkg/webhook/admission/servingruntime/servingruntime_webhook_test.go b/pkg/webhook/admission/servingruntime/servingruntime_webhook_test.go
new file mode 100644
--- /dev/null
+++ b/pkg/webhook/admission/servingruntime/servingruntime_webhook_test.go
@@ -0,0 +1,1348 @@
+/*
+Copyright 2023 The KServe Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package servingruntime
+
+import (
+	"fmt"
+	"github.com/golang/protobuf/proto"
+	"github.com/kserve/kserve/pkg/apis/serving/v1alpha1"
+	"github.com/kserve/kserve/pkg/constants"
+	"github.com/onsi/gomega"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"testing"
+)
+
+func TestValidateServingRuntimePriority(t *testing.T) {
+	scenarios := map[string]struct {
+		name                   string
+		newServingRuntime      *v1alpha1.ServingRuntime
+		existingServingRuntime *v1alpha1.ServingRuntime
+		expected               gomega.OmegaMatcher
+	}{
+		"When both serving runtimes are not MMS it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "new-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "existing-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(true),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When priority is same for model format in multi model serving runtime then it should return error": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "new-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(true),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "existing-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(true),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.3.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.Equal(fmt.Errorf(InvalidPriorityError, "sklearn")),
+		},
+		"When existing serving runtime is disabled it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "new-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "existing-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(true),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When new serving runtime and existing runtime are same it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When model format is same and supported protocol version is different it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When model format is different it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "lightgbm",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When autoselect is false in the new serving runtime it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(false),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When autoselect is not specified in the new serving runtime it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:     "sklearn",
+							Version:  proto.String("1"),
+							Priority: proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When autoselect is false in the existing serving runtime it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(false),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When model version is nil in both serving runtime and priority is same then it should return error": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.Equal(fmt.Errorf(InvalidPriorityError, "sklearn")),
+		},
+		"When model version is nil in both serving runtime and priority is not same then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(2),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When model version is nil in new serving runtime and priority is same then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When model version is nil in existing serving runtime and priority is same then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When model version is same in both serving runtime and priority is same then it should return error": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.Equal(fmt.Errorf(InvalidPriorityError, "sklearn")),
+		},
+		"When model version is different but priority is same then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1.3"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1.0"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When priority is nil in both serving runtime then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When priority is nil in new serving runtime and priority is specified in existing serving runtime then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+		"When priority is nil in existing serving runtime and priority is specified in new serving runtime then it should return nil": {
+			newServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-1",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+							Priority:   proto.Int32(1),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "kserve/sklearnserver:latest",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			existingServingRuntime: &v1alpha1.ServingRuntime{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "example-runtime-2",
+					Namespace: "test",
+				},
+				Spec: v1alpha1.ServingRuntimeSpec{
+					SupportedModelFormats: []v1alpha1.SupportedModelFormat{
+						{
+							Name:       "sklearn",
+							Version:    proto.String("1"),
+							AutoSelect: proto.Bool(true),
+						},
+					},
+					MultiModel: proto.Bool(false),
+					Disabled:   proto.Bool(false),
+					ProtocolVersions: []constants.InferenceServiceProtocol{
+						constants.ProtocolV1,
+						constants.ProtocolV2,
+					},
+					ServingRuntimePodSpec: v1alpha1.ServingRuntimePodSpec{
+						Containers: []corev1.Container{
+							{
+								Name:  constants.InferenceServiceContainerName,
+								Image: "seldonio/mlserver:1.2.0",
+								Args: []string{
+									"--model_name={{.Name}}",
+									"--model_dir=/mnt/models",
+									"--http_port=8080",
+								},
+							},
+						},
+					},
+				},
+			},
+			expected: gomega.BeNil(),
+		},
+	}
+
+	for name, scenario := range scenarios {
+		t.Run(name, func(t *testing.T) {
+			g := gomega.NewGomegaWithT(t)
+			err := validateServingRuntimePriority(&scenario.newServingRuntime.Spec, &scenario.existingServingRuntime.Spec,
+				scenario.newServingRuntime.Name, scenario.existingServingRuntime.Name)
+			g.Expect(err).To(scenario.expected)
+		})
+	}
+}
diff --git a/test/crds/serving.kserve.io_inferenceservices.yaml b/test/crds/serving.kserve.io_inferenceservices.yaml
--- a/test/crds/serving.kserve.io_inferenceservices.yaml
+++ b/test/crds/serving.kserve.io_inferenceservices.yaml
@@ -1131,6 +1131,10 @@ spec:
                       type: boolean
                     name:
                       type: string
+                    priority:
+                      format: int32
+                      minimum: 1
+                      type: integer
                     version:
                       type: string
                   required:
@@ -19595,6 +19599,10 @@ spec:
                       type: boolean
                     name:
                       type: string
+                    priority:
+                      format: int32
+                      minimum: 1
+                      type: integer
                     version:
                       type: string
                   required: