Repository: kubernetes-sigs/inference-perf
Pull request #71: Automatically Populate Missing Fields in Config
Author: login: Bslabe123
Labels: name: size/M
description: Denotes a PR that changes 30-99 lines, ignoring generated files.
name: lgtm
description: "Looks good to me", indicates that a PR is ready to be merged.
name: approved
description: Indicates a PR has been approved by an approver from all required OWNERS files.
name: cncf-cla: yes
description: Indicates the PR's author has signed the CNCF CLA.
Created: 2025-05-08T16:12:51Z | Merged: 2025-05-10T00:11:13Z

Old behavior was to fail if fields were omitted, new behavior is to fall back to defaults. Also removes the need for None type checks. Also logs the final merged config.

Other changes:
- Removed unused `request_processor.py` and `response_processor.py`


Diff:
diff --git a/inference_perf/client/request_processor.py b/inference_perf/client/request_processor.py
deleted file mode 100644
index 67592987..00000000
--- a/inference_perf/client/request_processor.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# Copyright 2025 The Kubernetes Authors.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
diff --git a/inference_perf/client/response_processor.py b/inference_perf/client/response_processor.py
deleted file mode 100644
index 67592987..00000000
--- a/inference_perf/client/response_processor.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# Copyright 2025 The Kubernetes Authors.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
diff --git a/inference_perf/config.py b/inference_perf/config.py
index 1fd68ea6..39c3c75a 100644
--- a/inference_perf/config.py
+++ b/inference_perf/config.py
@@ -13,7 +13,7 @@
 # limitations under the License.
 from datetime import datetime
 from pydantic import BaseModel
-from typing import Optional, List
+from typing import Any, Optional, List
 from argparse import ArgumentParser
 from enum import Enum
 import yaml
@@ -44,14 +44,14 @@ class MetricsClientType(Enum):
 
 
 class LoadStage(BaseModel):
-    rate: int = 1
-    duration: int = 1
+    rate: int
+    duration: int
 
 
 class LoadConfig(BaseModel):
     type: LoadType = LoadType.CONSTANT
-    interval: Optional[float] = 1.0
-    stages: List[LoadStage]
+    interval: float = 1.0
+    stages: List[LoadStage] = []
 
 
 class StorageConfigBase(BaseModel):
@@ -72,7 +72,7 @@ class ReportConfig(BaseModel):
 
 
 class PrometheusClientConfig(BaseModel):
-    scrape_interval: Optional[int] = 15
+    scrape_interval: int = 15
     url: str = "http://localhost:9090"
 
 
@@ -94,8 +94,8 @@ class CustomTokenizerConfig(BaseModel):
 
 
 class Config(BaseModel):
-    data: Optional[DataConfig] = DataConfig()
-    load: Optional[LoadConfig] = LoadConfig(stages=[LoadStage()])
+    data: DataConfig = DataConfig()
+    load: LoadConfig = LoadConfig()
     report: Optional[ReportConfig] = ReportConfig()
     metrics_client: Optional[MetricsClientConfig] = None
     storage: Optional[StorageConfig] = StorageConfig()
@@ -103,6 +103,16 @@ class Config(BaseModel):
     tokenizer: Optional[CustomTokenizerConfig] = None
 
 
+def deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
+    result = base.copy()
+    for k, v in override.items():
+        if k in result and isinstance(result[k], dict) and isinstance(v, dict):
+            result[k] = deep_merge(result[k], v)
+        else:
+            result[k] = v
+    return result
+
+
 def read_config() -> Config:
     parser = ArgumentParser()
 
@@ -110,10 +120,15 @@ def read_config() -> Config:
 
     args = parser.parse_args()
     if args.config_file:
-        print("Using configuration from: % s" % args.config_file)
+        print("Using configuration from: %s" % args.config_file)
         with open(args.config_file, "r") as stream:
             cfg = yaml.safe_load(stream)
 
-        return Config(**cfg)
+        default_cfg = Config().model_dump(mode="json")
+        merged_cfg = deep_merge(default_cfg, cfg)
 
+        print(
+            f"Benchmarking with the following config:\n\n{yaml.dump(merged_cfg, sort_keys=False, default_flow_style=False)}\n"
+        )
+        return Config(**merged_cfg)
     return Config()


Reviews:
user: login: SachinVarghese
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T00:44:18Z
user: login: SachinVarghese
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T01:13:53Z
user: login: Bslabe123
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T17:06:42Z
user: login: SachinVarghese
body: /lgtm
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-10T00:10:04Z

Review comments:
diff_hunk: @@ -94,26 +94,41 @@ class CustomTokenizerConfig(BaseModel):
 
 
 class Config(BaseModel):
-    data: Optional[DataConfig] = DataConfig()
-    load: Optional[LoadConfig] = LoadConfig(stages=[LoadStage()])
+    data: DataConfig = DataConfig()
+    load: LoadConfig = LoadConfig()
     report: Optional[ReportConfig] = ReportConfig()
     metrics_client: Optional[MetricsClientConfig] = None
     storage: Optional[StorageConfig] = StorageConfig()
     vllm: Optional[VLLMConfig] = None
     tokenizer: Optional[CustomTokenizerConfig] = None
 
 
+def deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
path: inference_perf/config.py
user: login: SachinVarghese
body: move to `utils` file?
created_at: 2025-05-09T00:43:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -94,26 +94,41 @@ class CustomTokenizerConfig(BaseModel):
 
 
 class Config(BaseModel):
-    data: Optional[DataConfig] = DataConfig()
-    load: Optional[LoadConfig] = LoadConfig(stages=[LoadStage()])
+    data: DataConfig = DataConfig()
+    load: LoadConfig = LoadConfig()
     report: Optional[ReportConfig] = ReportConfig()
     metrics_client: Optional[MetricsClientConfig] = None
     storage: Optional[StorageConfig] = StorageConfig()
     vllm: Optional[VLLMConfig] = None
     tokenizer: Optional[CustomTokenizerConfig] = None
 
 
+def deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
+    result = base.copy()
+    for k, v in override.items():
+        if k in result and isinstance(result[k], dict) and isinstance(v, dict):
+            result[k] = deep_merge(result[k], v)
+        else:
+            result[k] = v
+    return result
+
+
 def read_config() -> Config:
     parser = ArgumentParser()
 
     parser.add_argument("-c", "--config_file", help="Config File", required=True)
 
     args = parser.parse_args()
     if args.config_file:
-        print("Using configuration from: % s" % args.config_file)
+        print("Using configuration from: %s" % args.config_file)
         with open(args.config_file, "r") as stream:
             cfg = yaml.safe_load(stream)
 
-        return Config(**cfg)
+        default_cfg = Config().model_dump(mode="json")
path: inference_perf/config.py
user: login: SachinVarghese
body: how does mode="json" work with enums?
created_at: 2025-05-09T01:13:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -94,26 +94,41 @@ class CustomTokenizerConfig(BaseModel):
 
 
 class Config(BaseModel):
-    data: Optional[DataConfig] = DataConfig()
-    load: Optional[LoadConfig] = LoadConfig(stages=[LoadStage()])
+    data: DataConfig = DataConfig()
+    load: LoadConfig = LoadConfig()
     report: Optional[ReportConfig] = ReportConfig()
     metrics_client: Optional[MetricsClientConfig] = None
     storage: Optional[StorageConfig] = StorageConfig()
     vllm: Optional[VLLMConfig] = None
     tokenizer: Optional[CustomTokenizerConfig] = None
 
 
+def deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
+    result = base.copy()
+    for k, v in override.items():
+        if k in result and isinstance(result[k], dict) and isinstance(v, dict):
+            result[k] = deep_merge(result[k], v)
+        else:
+            result[k] = v
+    return result
+
+
 def read_config() -> Config:
     parser = ArgumentParser()
 
     parser.add_argument("-c", "--config_file", help="Config File", required=True)
 
     args = parser.parse_args()
     if args.config_file:
-        print("Using configuration from: % s" % args.config_file)
+        print("Using configuration from: %s" % args.config_file)
         with open(args.config_file, "r") as stream:
             cfg = yaml.safe_load(stream)
 
-        return Config(**cfg)
+        default_cfg = Config().model_dump(mode="json")
path: inference_perf/config.py
user: login: Bslabe123
body: Without `mode="json`, `type: !!python/object/apply:inference_perf.config.LoadType` would be printed instead of `type: constant`
created_at: 2025-05-09T17:06:42Z
author_association: CONTRIBUTOR

PR comments:
user: login: k8s-ci-robot
created_at: 2025-05-10T00:10:10Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/inference-perf/pull/71#" title="Author self-approved">Bslabe123</a>*, *<a href="https://github.com/kubernetes-sigs/inference-perf/pull/71#pullrequestreview-2829955558" title="LGTM">SachinVarghese</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Finference-perf).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/inference-perf/blob/main/OWNERS)~~ [SachinVarghese]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: CONTRIBUTOR