Repository: truffle-ai/saiki
Pull request #85: Code changes to support model config as script param
Author: login: rahulkarajgikar
Created: 2025-05-01T04:08:49Z | Merged: 2025-05-02T07:40:08Z

 - feature: adding model/router arguments to CLI commands with dynamic provider inference
 - feature: added a utility to compare zod schemas with typescript interfaces that they represent - to keep schemas and interface in sync `zod.ts`. useful for codebase cleanliness.
 - feature: added basic redaction logic in logger to hide any API keys if detected in logs. right now just using basic regex checks but can be improved if needed
 - feature: new ConfigManager service: agent config management to a new file `src/config/manager.ts` as a service. keeping it as a first class service is an initial step for us to enable dynamic config changes later on.
 - centralized all supported LLM model metadata and token counts to one file `llm/registry.ts`. (just one code change to add a new model/provider now) - better maintainability and easier changes
 - simplified application start logic in `index.ts` - moved away CLI options validation and config validation to separate files. created new function `createAgentServices` this is a step towards cleanly separating core layer and exposing saiki as an sdk

    - Priority for any setting:  CLI overrides > config file > defaults


resolves #84 and allows us to easily add more options via CLI

Linked issue #84: Add an option to change model dynamically during CLI start and runtime
A command like `saiki -m gemini-2.0-flash` would be easier to use to toggle models instead of editing the config yml each time. 
Linked issue comments: user: login: AayushSaini101
created_at: 2025-05-01T13:06:40Z
body: @shaunak99 i would like to work on this issue thanks
author_association: CONTRIBUTOR
user: login: rahulkarajgikar
created_at: 2025-05-01T21:27:45Z
body: @AayushSaini101 i am already working on this as part of #85, sorry i should have marked it as assigned to me. have just assigned it. Feel free to check other issues! 

Will make sure to assign issues that are already being worked on to avoid confusion

author_association: COLLABORATOR

Diff:
diff --git a/app/index.ts b/app/index.ts
index ee3c556c0..5c4012d8b 100644
--- a/app/index.ts
+++ b/app/index.ts
@@ -3,12 +3,12 @@ import { existsSync } from 'fs';
 import { Command } from 'commander';
 import dotenv from 'dotenv';
 import { logger } from '../src/utils/logger.js';
-import { loadConfigFile } from '../src/config/loader.js';
 import { DEFAULT_CONFIG_PATH, resolvePackagePath } from '../src/utils/path.js';
-import { AgentConfig } from '../src/config/types.js';
-import { initializeServices } from '../src/utils/service-initializer.js';
+import { createAgentServices } from '../src/utils/service-initializer.js';
 import { runAiCli } from './cli/cli.js';
 import { initializeWebUI } from './web/server.js';
+import { validateCliOptions, handleCliOptionsError } from '../src/utils/options.js';
+import { getProviderFromModel, getAllSupportedModels } from '../src/ai/llm/registry.js';
 
 // Load environment variables
 dotenv.config();
@@ -49,38 +49,67 @@ if (
 program
     .name('saiki')
     .description('AI-powered CLI and WebUI for interacting with MCP servers')
+    // General Options
     .option('-c, --config-file <path>', 'Path to config file', DEFAULT_CONFIG_PATH)
     .option('-s, --strict', 'Require all server connections to succeed')
     .option('--no-verbose', 'Disable verbose output')
     .option('--mode <mode>', 'Run mode: cli or web', 'cli')
     .option('--web-port <port>', 'Port for WebUI', '3000')
+    // LLM Options
+    .option('-m, --model <model>', 'Specify the LLM model to use')
+    .option('-r, --router <router>', 'Specify the LLM router to use (vercel or in-built)')
     .version('0.2.0');
 
 program.parse();
 
 // Get options
 const options = program.opts();
+// Dynamically infer provider and api key from the supplied model
+if (options.model) {
+    let modelProvider: string;
+    try {
+        modelProvider = getProviderFromModel(options.model);
+    } catch (err) {
+        // Model inference failed
+        const msg = err instanceof Error ? err.message : String(err);
+        logger.error(`ERROR: ${msg}`);
+        logger.error(`Supported models are:\n${getAllSupportedModels().join(', ')}`);
+        process.exit(1);
+    }
+    options.provider = modelProvider;
+
+    // Dynamically extract the actual API key for the provider
+    const providerEnvMap: Record<string, string> = {
+        openai: 'OPENAI_API_KEY',
+        anthropic: 'ANTHROPIC_API_KEY',
+        google: 'GOOGLE_GENERATIVE_AI_API_KEY',
+    };
+    const envVarName = providerEnvMap[modelProvider];
+    if (envVarName) {
+        const key = process.env[envVarName];
+        if (!key) {
+            logger.error(`ERROR: Missing ${envVarName} environment variable for provider '${modelProvider}'.`);
+            process.exit(1);
+        }
+        options.apiKey = key;
+    }
+}
+
 const configFile = options.configFile;
 const connectionMode = options.strict ? 'strict' : ('lenient' as 'strict' | 'lenient');
 const runMode = options.mode.toLowerCase();
 const webPort = parseInt(options.webPort, 10);
 const resolveFromPackageRoot = configFile === DEFAULT_CONFIG_PATH; // Check if should resolve from package root
-
-// Validate run mode
-if (!['cli', 'web'].includes(runMode)) {
-    logger.error(`Invalid mode: ${runMode}. Must be 'cli' or 'web'.`);
-    process.exit(1);
-}
-
-// Validate web port
-if (isNaN(webPort) || webPort <= 0 || webPort > 65535) {
-    logger.error(`Invalid web port: ${options.webPort}. Must be a number between 1 and 65535.`);
-    process.exit(1);
-}
-
 // Platform-independent path handling
 const normalizedConfigPath = resolvePackagePath(configFile, resolveFromPackageRoot);
 
+// basic validation of options here
+try {
+    validateCliOptions(options);
+} catch (error) {
+    handleCliOptionsError(error);
+}
+
 logger.info(`Initializing Saiki with config: ${normalizedConfigPath}`, null, 'blue');
 
 // Conditionally display CLI examples
@@ -97,42 +126,39 @@ if (runMode === 'cli') {
 
 // Main start function
 async function startAgent() {
+    // Use createAgentServices to load, validate config and initialize all agent services
+    const cliArgs = { model: options.model, provider: options.provider, router: options.router, apiKey: options.apiKey };
+    let services;
     try {
-        // Load the agent configuration
-        const config: AgentConfig = await loadConfigFile(normalizedConfigPath);
-        validateAgentConfig(config);
-
-        logger.info('===============================================');
-        logger.info(`Initializing Saiki in '${runMode}' mode...`, null, 'cyanBright');
-        logger.info('===============================================\n');
-
-        // Use the shared initializer
-        const { clientManager, llmService, agentEventBus } = await initializeServices(config, {
+        services = await createAgentServices(normalizedConfigPath, cliArgs, {
             connectionMode,
             runMode,
         });
-
-        // Start based on mode
-        if (runMode === 'cli') {
-            // Run CLI
-            await runAiCli(clientManager, llmService, agentEventBus);
-        } else if (runMode === 'web') {
-            // Run WebUI
-            initializeWebUI(clientManager, llmService, agentEventBus, webPort);
-            logger.info(`WebUI available at http://localhost:${webPort}`, null, 'magenta');
-            // Note: Web server runs indefinitely, no need to await here unless
-            // you specifically want the script to only exit when the server does.
+    } catch (err) {
+        if (err instanceof Error) {
+            err.message.split('\n').forEach((line) => logger.error(line));
+        } else {
+            logger.error('Unexpected error during startup:', err);
         }
-    } catch (error) {
-        logger.error(
-            `Error: Failed to initialize AI CLI from config file ${normalizedConfigPath}: ${
-                error instanceof Error
-                    ? `${error.message}\n${error.stack}`
-                    : JSON.stringify(error, null, 2)
-            }`
-        );
         process.exit(1);
     }
+
+    logger.info('===============================================');
+    logger.info(`Initializing Saiki in '${runMode}' mode...`, null, 'cyanBright');
+    logger.info('===============================================\n');
+
+    // Destructure the agent runtime services
+    const { clientManager, llmService, agentEventBus } = services;
+
+    // Start based on mode
+    if (runMode === 'cli') {
+        // Run CLI
+        await runAiCli(clientManager, llmService, agentEventBus);
+    } else if (runMode === 'web') {
+        // Run WebUI
+        initializeWebUI(clientManager, llmService, agentEventBus, webPort);
+        logger.info(`WebUI available at http://localhost:${webPort}`, null, 'magenta');
+    }
 }
 
 // Execute the agent
@@ -141,38 +167,3 @@ startAgent().catch((error) => {
     logger.error(error);
     process.exit(1);
 });
-
-function validateAgentConfig(config: AgentConfig): void {
-    logger.info('Validating agent config', 'cyanBright');
-    if (!config.mcpServers || Object.keys(config.mcpServers).length === 0) {
-        throw new Error('No MCP server configurations provided in config file.');
-    }
-
-    // Validate LLM section exists, use defaults if not
-    if (!config.llm) {
-        logger.info('No LLM configuration found, applying defaults', 'yellow');
-        config.llm = {
-            provider: 'openai',
-            model: 'gpt-4.1-mini',
-            systemPrompt:
-                'You are Saiki, a helpful AI assistant with access to tools. Use these tools when appropriate to answer user queries. You can use multiple tools in sequence to solve complex problems. After each tool result, determine if you need more information or can provide a final answer.',
-            apiKey: '$OPENAI_API_KEY',
-        };
-    } else {
-        // Ensure required LLM fields are present if section exists
-        if (!config.llm.provider || !config.llm.model) {
-            throw new Error('LLM configuration must specify provider and model');
-        }
-        // Provide default system prompt if missing
-        if (!config.llm.systemPrompt) {
-            logger.info('No system prompt found, using default', 'yellow');
-            config.llm.systemPrompt =
-                'You are Saiki, a helpful AI assistant with access to tools. Use these tools when appropriate to answer user queries. You can use multiple tools in sequence to solve complex problems. After each tool result, determine if you need more information or can provide a final answer.';
-        }
-    }
-
-    logger.info(
-        `Found ${Object.keys(config.mcpServers).length} server configurations. Validation successful.`,
-        'green'
-    );
-}
diff --git a/package-lock.json b/package-lock.json
index 9580441d2..724033637 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,11 +1,11 @@
 {
-    "name": "saiki",
+    "name": "@truffle-ai/saiki",
     "version": "0.2.1",
     "lockfileVersion": 3,
     "requires": true,
     "packages": {
         "": {
-            "name": "saiki",
+            "name": "@truffle-ai/saiki",
             "version": "0.2.1",
             "license": "Elastic-2.0",
             "dependencies": {
@@ -32,7 +32,7 @@
                 "winston": "^3.17.0",
                 "ws": "^8.18.1",
                 "yaml": "^2.7.1",
-                "zod": "^3.22.4"
+                "zod": "^3.24.3"
             },
             "bin": {
                 "saiki": "dist/app/index.js"
@@ -6171,9 +6171,10 @@
             }
         },
         "node_modules/zod": {
-            "version": "3.24.2",
-            "resolved": "https://registry.npmjs.org/zod/-/zod-3.24.2.tgz",
-            "integrity": "sha512-lY7CDW43ECgW9u1TcT3IoXHflywfVqDYze4waEz812jR/bZ8FHDsl7pFQoSZTz5N+2NqRXs8GBwnAwo3ZNxqhQ==",
+            "version": "3.24.3",
+            "resolved": "https://registry.npmjs.org/zod/-/zod-3.24.3.tgz",
+            "integrity": "sha512-HhY1oqzWCQWuUqvBFnsyrtZRhyPeR7SUGv+C4+MsisMuVfSPx8HpwWqH8tRahSlt6M3PiFAcoeFhZAqIXTxoSg==",
+            "license": "MIT",
             "funding": {
                 "url": "https://github.com/sponsors/colinhacks"
             }
diff --git a/package.json b/package.json
index 905d7006e..780e03071 100644
--- a/package.json
+++ b/package.json
@@ -60,7 +60,7 @@
         "winston": "^3.17.0",
         "ws": "^8.18.1",
         "yaml": "^2.7.1",
-        "zod": "^3.22.4"
+        "zod": "^3.24.3"
     },
     "devDependencies": {
         "@eslint/js": "^9.23.0",
diff --git a/src/ai/llm/messages/factory.ts b/src/ai/llm/messages/factory.ts
index d695b6164..ce3bb8718 100644
--- a/src/ai/llm/messages/factory.ts
+++ b/src/ai/llm/messages/factory.ts
@@ -1,21 +1,20 @@
 import { MessageManager } from './manager.js';
-import { VercelMessageFormatter } from './formatters/vercel.js';
-import { OpenAIMessageFormatter } from './formatters/openai.js';
-import { AnthropicMessageFormatter } from './formatters/anthropic.js';
+import { createMessageFormatter } from './formatters/factory.js';
 import { createTokenizer } from '../tokenizer/factory.js';
-import { getMaxTokens } from '../tokenizer/utils.js';
 import { LLMConfig } from '../../../config/types.js';
 import { LLMRouter } from '../types.js';
 import { IMessageFormatter } from './formatters/types.js';
 import { ITokenizer } from '../tokenizer/types.js';
 import { SystemPromptContributor } from '../../systemPrompt/types.js';
+import { logger } from '../../../utils/logger.js';
+import { getMaxTokensForModel } from '../registry.js';
 
 /**
  * Factory function to create a MessageManager instance with the correct formatter, tokenizer, and maxTokens
- * based on the LLM config and router (vercel, default, or future types).
+ * based on the LLM config and router
  *
  * @param config LLMConfig object containing provider, model, systemPrompt, etc.
- * @param router LLMRouter flag ('vercel', 'default', etc.)
+ * @param router LLMRouter flag
  * @param contributors SystemPromptContributor[]
  * @returns MessageManager instance
  * TODO: Make compression strategy also configurable
@@ -25,31 +24,20 @@ export function createMessageManager(
     router: LLMRouter = 'vercel',
     contributors: SystemPromptContributor[]
 ): MessageManager {
-    let formatter: IMessageFormatter;
-    let tokenizer: ITokenizer;
-    let maxTokens: number;
     const provider = config.provider.toLowerCase();
-    const model = config.model;
+    const model = config.model.toLowerCase();
 
-    if (router === 'vercel') {
-        formatter = new VercelMessageFormatter();
-        tokenizer = createTokenizer(provider, model);
-        maxTokens = Math.floor(getMaxTokens(provider, model) * 0.9);
-    } else if (router === 'default') {
-        if (provider === 'openai') {
-            formatter = new OpenAIMessageFormatter();
-            tokenizer = createTokenizer('openai', model);
-            maxTokens = Math.floor(getMaxTokens('openai', model) * 0.9);
-        } else if (provider === 'anthropic') {
-            formatter = new AnthropicMessageFormatter();
-            tokenizer = createTokenizer('anthropic', model);
-            maxTokens = Math.floor(getMaxTokens('anthropic', model) * 0.9);
-        } else {
-            throw new Error(`Unsupported LLM provider: ${provider} for router: ${router}`);
-        }
-    } else {
-        throw new Error(`Unsupported LLM router: ${router}`);
-    }
+    const registryMaxTokens = getMaxTokensForModel(provider, model);
+    const maxTokens = Math.floor(registryMaxTokens! * 0.9);
 
+    const tokenizer = createTokenizer(provider, model);
+    logger.debug(`Tokenizer created for ${provider}/${model}`);
+
+    const formatter = createMessageFormatter(provider, router);
+    logger.debug(`Message formatter created for ${provider}/${model}`);
+
+    logger.debug(
+        `Creating MessageManager for ${provider}/${model} using ${router} router with maxTokens: ${maxTokens}`
+    );
     return new MessageManager(formatter, contributors, maxTokens, tokenizer);
 }
diff --git a/src/ai/llm/messages/formatters/factory.ts b/src/ai/llm/messages/formatters/factory.ts
new file mode 100644
index 000000000..287737d36
--- /dev/null
+++ b/src/ai/llm/messages/formatters/factory.ts
@@ -0,0 +1,25 @@
+import { IMessageFormatter } from './types.js';
+import { VercelMessageFormatter } from './vercel.js';
+import { OpenAIMessageFormatter } from './openai.js';
+import { AnthropicMessageFormatter } from './anthropic.js';
+import { logger } from '../../../../utils/logger.js';
+
+export function createMessageFormatter(provider: string, router: string): IMessageFormatter {
+    if (router === 'vercel') {
+        return new VercelMessageFormatter();
+    } else if (router === 'in-built') {
+        if (provider === 'openai') {
+            return new OpenAIMessageFormatter();
+        } else if (provider === 'anthropic') {
+            return new AnthropicMessageFormatter();
+        } else {
+            logger.error(
+                `Provider '${provider}' supported by registry but not configured for 'default' router message formatting.`
+            );
+            throw new Error(`Unsupported LLM provider: ${provider} for router: ${router}`);
+        }
+    } else {
+        logger.error(`Unsupported LLM router specified: ${router}`);
+        throw new Error(`Unsupported LLM router: ${router}`);
+    }
+}
diff --git a/src/ai/llm/registry.ts b/src/ai/llm/registry.ts
new file mode 100644
index 000000000..dab841538
--- /dev/null
+++ b/src/ai/llm/registry.ts
@@ -0,0 +1,150 @@
+import { logger } from '../../utils/logger.js';
+
+export interface ModelInfo {
+    name: string;
+    maxTokens: number;
+    // Add other relevant metadata if needed, e.g., supported features, cost tier
+}
+
+export interface ProviderInfo {
+    models: ModelInfo[];
+    // Add other provider-specific metadata if needed
+}
+
+// Central registry of supported LLM providers and their models
+export const LLM_REGISTRY: Record<string, ProviderInfo> = {
+    openai: {
+        models: [
+            { name: 'gpt-4.1', maxTokens: 1047576 },
+            { name: 'gpt-4.1-mini', maxTokens: 1047576 },
+            { name: 'gpt-4.1-nano', maxTokens: 1047576 },
+            { name: 'gpt-4o', maxTokens: 128000 },
+            { name: 'gpt-4o-mini', maxTokens: 128000 },
+            { name: 'o4-mini', maxTokens: 200000 },
+            { name: 'o3', maxTokens: 200000 },
+            { name: 'o3-mini', maxTokens: 200000 },
+            { name: 'o1', maxTokens: 200000 },
+        ],
+    },
+    anthropic: {
+        models: [
+            { name: 'claude-3-7-sonnet-20250219', maxTokens: 200000 },
+            { name: 'claude-3-5-sonnet-20240620', maxTokens: 200000 },
+            { name: 'claude-3-haiku-20240307', maxTokens: 200000 },
+            { name: 'claude-3-opus-20240229', maxTokens: 200000 },
+            { name: 'claude-3-sonnet-20240229', maxTokens: 200000 },
+        ],
+    },
+    google: {
+        models: [
+            { name: 'gemini-2.5-pro', maxTokens: 1048576 },
+            { name: 'gemini-2.0-flash', maxTokens: 1048576 },
+            { name: 'gemini-2.0-flash-lite', maxTokens: 1048576 },
+            { name: 'gemini-1.5-pro-latest', maxTokens: 1048576 },
+            { name: 'gemini-1.5-flash-latest', maxTokens: 1048576 },
+        ],
+    },
+    // Add other providers like Groq, Cohere, etc., as needed
+};
+
+/**
+ * Gets the list of supported providers.
+ * @returns An array of supported provider names.
+ */
+export function getSupportedProviders(): string[] {
+    return Object.keys(LLM_REGISTRY);
+}
+
+/**
+ * Gets the list of supported models for a given provider.
+ * @param provider The name of the provider.
+ * @returns An array of supported model names for the provider, or an empty array if the provider is not found.
+ */
+export function getSupportedModels(provider: string): string[] {
+    const providerInfo = LLM_REGISTRY[provider.toLowerCase()];
+    return providerInfo ? providerInfo.models.map((m) => m.name) : [];
+}
+
+/**
+ * Retrieves the maximum token limit for a given provider and model from the registry.
+ * Throws an error if the provider or model is not found.
+ * @param provider The name of the provider (e.g., 'openai', 'anthropic', 'google').
+ * @param model The specific model name.
+ * @returns The maximum token limit for the model.
+ * @throws {Error} If the provider or model is not found in the registry.
+ */
+export function getMaxTokensForModel(provider: string, model: string): number {
+    const lowerProvider = provider?.toLowerCase();
+    const lowerModel = model?.toLowerCase();
+
+    const providerInfo = LLM_REGISTRY[lowerProvider];
+    if (!providerInfo) {
+        const supportedProviders = getSupportedProviders().join(', ');
+        logger.error(
+            `Provider '${provider}' not found in LLM registry. Supported: ${supportedProviders}`
+        );
+        throw new Error(
+            `Provider '${provider}' not found in LLM registry. Supported providers are: ${supportedProviders}`
+        );
+    }
+
+    const modelInfo = providerInfo.models.find((m) => m.name.toLowerCase() === lowerModel);
+    if (!modelInfo) {
+        const supportedModels = getSupportedModels(lowerProvider).join(', ');
+        logger.error(
+            `Model '${model}' not found for provider '${provider}' in LLM registry. Supported models: ${supportedModels}`
+        );
+        throw new Error(
+            `Model '${model}' not found for provider '${provider}' in LLM registry. Supported models for '${provider}' are: ${supportedModels}`
+        );
+    }
+
+    logger.debug(`Found max tokens for ${provider}/${model}: ${modelInfo.maxTokens}`);
+    return modelInfo.maxTokens;
+}
+
+/**
+ * Validates if a provider and model combination is supported.
+ * @param provider The provider name.
+ * @param model The model name.
+ * @returns True if the combination is valid, false otherwise.
+ */
+export function isValidProviderModel(provider?: string, model?: string): boolean {
+    if (!provider || !model) {
+        // If either is missing, we consider it valid at this level
+        // (the refine function in Zod handles the both-or-neither case)
+        return true;
+    }
+    const lowerProvider = provider.toLowerCase();
+    const lowerModel = model.toLowerCase();
+    const providerInfo = LLM_REGISTRY[lowerProvider];
+    if (!providerInfo) {
+        return false;
+    }
+    return providerInfo.models.some((m) => m.name === lowerModel);
+}
+
+/**
+ * Infers the LLM provider from the model name by searching the registry.
+ * Matches the model name (case-insensitive) against all registered models.
+ * Returns the provider name if found, or 'unknown' if not found.
+ *
+ * @param model The model name (e.g., 'gpt-4o-mini', 'claude-3-7-sonnet-20250219')
+ * @returns The inferred provider name ('openai', 'anthropic', etc.), or 'unknown' if no match is found.
+ */
+export function getProviderFromModel(model: string): string {
+    const lowerModel = model.toLowerCase();
+    for (const [provider, info] of Object.entries(LLM_REGISTRY)) {
+        if (info.models.some((m) => m.name.toLowerCase() === lowerModel)) {
+            return provider;
+        }
+    }
+    throw new Error(`Unrecognized model '${model}'. Could not infer provider.`);
+}
+
+/**
+ * Returns a flat array of all supported model names from all providers.
+ */
+export function getAllSupportedModels(): string[] {
+    return Object.values(LLM_REGISTRY).flatMap((info) => info.models.map((m) => m.name));
+}
diff --git a/src/ai/llm/services/anthropic.ts b/src/ai/llm/services/anthropic.ts
index a5d3ffd61..bf9b5fde5 100644
--- a/src/ai/llm/services/anthropic.ts
+++ b/src/ai/llm/services/anthropic.ts
@@ -5,7 +5,7 @@ import { ToolSet } from '../../types.js';
 import { logger } from '../../../utils/logger.js';
 import { EventEmitter } from 'events';
 import { MessageManager } from '../messages/manager.js';
-import { getMaxTokens } from '../tokenizer/utils.js';
+import { getMaxTokensForModel } from '../registry.js';
 import { ImageData } from '../messages/types.js';
 
 /**
@@ -200,11 +200,13 @@ export class AnthropicService implements ILLMService {
      */
     getConfig(): LLMServiceConfig {
         const configuredMaxTokens = this.messageManager.getMaxTokens();
+
         return {
+            router: 'in-built',
             provider: 'anthropic',
             model: this.model,
             configuredMaxTokens: configuredMaxTokens,
-            modelMaxTokens: getMaxTokens('anthropic', this.model),
+            modelMaxTokens: getMaxTokensForModel('anthropic', this.model),
         };
     }
 
diff --git a/src/ai/llm/services/factory.ts b/src/ai/llm/services/factory.ts
index b9f4377d5..cb4d79082 100644
--- a/src/ai/llm/services/factory.ts
+++ b/src/ai/llm/services/factory.ts
@@ -55,7 +55,7 @@ function _createInBuiltLLMService(
 ): ILLMService {
     // Extract and validate API key
     const apiKey = extractApiKey(config);
-    
+
     switch (config.provider.toLowerCase()) {
         case 'openai': {
             const openai = new OpenAI({ apiKey });
@@ -104,12 +104,17 @@ function _createVercelLLMService(
     messageManager: MessageManager
 ): VercelLLMService {
     const model: LanguageModelV1 = _createVercelModel(config.provider, config.model);
-    return new VercelLLMService(clientManager, model, agentEventBus, messageManager, config.maxIterations);
+    return new VercelLLMService(
+        clientManager,
+        model,
+        agentEventBus,
+        messageManager,
+        config.maxIterations
+    );
 }
 
 /**
  * Enum/type for LLM routing backend selection.
- * 'vercel' = use Vercel LLM service, 'default' = use in-built LLM service
  */
 export function createLLMService(
     config: LLMConfig,
diff --git a/src/ai/llm/services/openai.ts b/src/ai/llm/services/openai.ts
index fc6ccec35..a50a34616 100644
--- a/src/ai/llm/services/openai.ts
+++ b/src/ai/llm/services/openai.ts
@@ -5,7 +5,7 @@ import { ToolSet } from '../../types.js';
 import { logger } from '../../../utils/logger.js';
 import { EventEmitter } from 'events';
 import { MessageManager } from '../messages/manager.js';
-import { getMaxTokens } from '../tokenizer/utils.js';
+import { getMaxTokensForModel } from '../registry.js';
 import { ImageData } from '../messages/types.js';
 
 /**
@@ -162,11 +162,13 @@ export class OpenAIService implements ILLMService {
      */
     getConfig(): LLMServiceConfig {
         const configuredMaxTokens = this.messageManager.getMaxTokens();
+
         return {
+            router: 'in-built',
             provider: 'openai',
             model: this.model,
             configuredMaxTokens: configuredMaxTokens,
-            modelMaxTokens: getMaxTokens('openai', this.model),
+            modelMaxTokens: getMaxTokensForModel('openai', this.model),
         };
     }
 
diff --git a/src/ai/llm/services/types.ts b/src/ai/llm/services/types.ts
index 57409f3cd..3467a5721 100644
--- a/src/ai/llm/services/types.ts
+++ b/src/ai/llm/services/types.ts
@@ -38,6 +38,7 @@ export interface ILLMService {
  * Configuration object returned by LLMService.getConfig()
  */
 export type LLMServiceConfig = {
+    router: string;
     provider: string;
     model: string | LanguageModelV1;
     configuredMaxTokens?: number | null;
diff --git a/src/ai/llm/services/vercel.ts b/src/ai/llm/services/vercel.ts
index 311d7a47f..249a8bf8c 100644
--- a/src/ai/llm/services/vercel.ts
+++ b/src/ai/llm/services/vercel.ts
@@ -6,8 +6,8 @@ import { ToolSet } from '../../types.js';
 import { ToolSet as VercelToolSet, jsonSchema } from 'ai';
 import { EventEmitter } from 'events';
 import { MessageManager } from '../messages/manager.js';
-import { getProviderFromModel } from '../../utils.js';
-import { getMaxTokens } from '../tokenizer/utils.js';
+import { getProviderFromModel } from '../registry.js';
+import { getMaxTokensForModel } from '../registry.js';
 import { ImageData } from '../messages/types.js';
 
 /**
@@ -303,12 +303,14 @@ export class VercelLLMService implements ILLMService {
      * @returns Configuration object with provider and model information
      */
     getConfig(): LLMServiceConfig {
-        const configuredMaxTokens = (this.messageManager as any).maxTokens;
+        const configuredMaxTokens = this.messageManager.getMaxTokens();
+
         return {
-            provider: `vercel:${this.provider}`,
+            router: 'vercel',
+            provider: `${this.provider}`,
             model: this.model,
             configuredMaxTokens: configuredMaxTokens,
-            modelMaxTokens: getMaxTokens(this.provider, this.model.modelId),
+            modelMaxTokens: getMaxTokensForModel(this.provider, this.model.modelId),
         };
     }
 }
diff --git a/src/ai/llm/tokenizer/utils.ts b/src/ai/llm/tokenizer/utils.ts
deleted file mode 100644
index d1737da1f..000000000
--- a/src/ai/llm/tokenizer/utils.ts
+++ /dev/null
@@ -1,91 +0,0 @@
-import { logger } from '../../../utils/logger.js';
-
-const DEFAULT_TOKEN_LIMIT = 100000;
-
-// Map of known model token limits by provider and model substring
-const MODEL_TOKEN_LIMITS: Record<string, Record<string, number>> = {
-    openai: {
-        'gpt-4.1-nano': 1000000,
-        'gpt-4.1-mini': 1000000,
-        'gpt-4.1': 1000000,
-        'gpt-4o-mini': 128000,
-        'gpt-4o': 128000,
-        'o4-mini': 200000,
-    },
-    anthropic: {
-        'claude-3-7-sonnet': 200000,
-        'claude-3-5-haiku': 200000,
-        'claude-3-5-sonnet': 200000,
-        'claude-3-opus': 200000,
-        'claude-3-sonnet': 200000,
-        'claude-3-haiku': 200000,
-    },
-    google: {
-        'gemini-2.5-pro': 1048576,
-        'gemini-2.0-flash': 1048576,
-        'gemini-2.0-flash-lite': 1048576,
-        'gemini-1.5-pro': 2097152,
-        'gemini-1.5-flash': 1048576,
-    },
-    // Add other providers here as needed
-};
-
-// Provider-specific default token limits
-const PROVIDER_DEFAULTS: Record<string, number> = {
-    openai: 128000,
-    anthropic: 100000,
-    google: 1048576,
-};
-
-/**
- * Normalizes the model name by removing known suffixes such as dates or version identifiers.
- * @param modelName The original model name.
- * @returns The normalized model name.
- */
-function normalizeModelName(modelName: string): string {
-    return modelName
-        .toLowerCase()
-        .replace(/[@-]\d{6,8}$/, '') // Removes suffixes like '-20250219' or '@20250219'
-        .replace(/-\d{3}$/, ''); // Removes version suffixes like '-001'
-}
-
-/**
- * Retrieves the maximum token limit for a given provider and model.
- * @param provider The name of the provider (e.g., 'openai', 'anthropic', 'google').
- * @param model The specific model name.
- * @returns The maximum token limit for the model.
- */
-export function getMaxTokens(provider: string, model: string): number {
-    const lowerProvider = provider.toLowerCase();
-    const normalizedModel = normalizeModelName(model);
-    logger.debug(
-        `getMaxTokens: provider: ${provider}, model: ${model}, normalizedModel: ${normalizedModel}`
-    );
-
-    const providerModels = MODEL_TOKEN_LIMITS[lowerProvider];
-    if (providerModels) {
-        // Sort model keys by length in descending order to match more specific names first
-        const sortedModelKeys = Object.keys(providerModels).sort((a, b) => b.length - a.length);
-
-        for (const modelKey of sortedModelKeys) {
-            if (normalizedModel.includes(modelKey)) {
-                logger.debug(
-                    `getMaxTokens: found modelKey: ${modelKey}, limit: ${providerModels[modelKey]}`
-                );
-                return providerModels[modelKey];
-            }
-        }
-
-        logger.warn(
-            `Unknown ${provider} model: ${model}. Using default limit of ${
-                PROVIDER_DEFAULTS[lowerProvider] ?? DEFAULT_TOKEN_LIMIT
-            }.`
-        );
-        return PROVIDER_DEFAULTS[lowerProvider] ?? DEFAULT_TOKEN_LIMIT;
-    }
-
-    logger.warn(
-        `Unknown provider: ${provider} for model: ${model}. Using default limit of ${DEFAULT_TOKEN_LIMIT}.`
-    );
-    return DEFAULT_TOKEN_LIMIT;
-}
diff --git a/src/ai/llm/types.ts b/src/ai/llm/types.ts
index 44aeab150..e81e2d421 100644
--- a/src/ai/llm/types.ts
+++ b/src/ai/llm/types.ts
@@ -1,5 +1,5 @@
 /**
  * LLMRouter defines the routing backend for LLM service instantiation.
- * 'vercel' = use Vercel LLM service, 'default' = use in-built LLM service
+ * 'vercel' = use Vercel LLM service, 'in-built' = use in-built LLM service
  */
-export type LLMRouter = 'vercel' | 'default';
+export type LLMRouter = 'vercel' | 'in-built';
diff --git a/src/ai/utils.ts b/src/ai/utils.ts
deleted file mode 100644
index 2da13adf4..000000000
--- a/src/ai/utils.ts
+++ /dev/null
@@ -1,28 +0,0 @@
-import { logger } from '../utils/logger.js';
-
-const MODEL_PREFIX_TO_PROVIDER: Record<string, string> = {
-    'gpt-': 'openai',
-    'o4-': 'openai',
-    'claude-': 'anthropic',
-    'gemini-': 'google',
-    'grok-': 'grok',
-    // Add more as needed
-};
-
-/**
- * Infers the LLM provider from the model name using a prefix-to-provider map.
- * Matches the start of the model name against known prefixes for extensibility.
- *
- * @param model The model name (e.g., 'gpt-4o-mini', 'claude-3-7-sonnet-20250219')
- * @returns The inferred provider name ('openai', 'anthropic', etc.), or 'unknown' if no match is found.
- */
-export function getProviderFromModel(model: string): string {
-    const lowerModel = model.toLowerCase();
-    for (const prefix in MODEL_PREFIX_TO_PROVIDER) {
-        if (lowerModel.startsWith(prefix)) {
-            return MODEL_PREFIX_TO_PROVIDER[prefix];
-        }
-    }
-    logger.warn(`Could not determine provider for model: ${model}. Defaulting to 'unknown'.`);
-    return 'unknown';
-}
diff --git a/src/client/manager.ts b/src/client/manager.ts
index 14aa4aac4..50eba1539 100644
--- a/src/client/manager.ts
+++ b/src/client/manager.ts
@@ -40,13 +40,15 @@ export class MCPClientManager {
         const client = this.clients.get(clientName);
         if (!client) return;
 
-        [this.toolToClientMap, this.promptToClientMap, this.resourceToClientMap].forEach(cacheMap => {
-            for (const [key, mappedClient] of cacheMap.entries()) {
-                if (mappedClient === client) {
-                    cacheMap.delete(key);
+        [this.toolToClientMap, this.promptToClientMap, this.resourceToClientMap].forEach(
+            (cacheMap) => {
+                for (const [key, mappedClient] of cacheMap.entries()) {
+                    if (mappedClient === client) {
+                        cacheMap.delete(key);
+                    }
                 }
             }
-        });
+        );
         logger.debug(`Cleared cache for client: ${clientName}`);
     }
 
@@ -65,7 +67,7 @@ export class MCPClientManager {
         // Cache prompts, if supported
         try {
             const prompts = await client.listPrompts();
-            prompts.forEach(promptName => {
+            prompts.forEach((promptName) => {
                 this.promptToClientMap.set(promptName, client);
             });
             logger.debug(`Cached prompts for client: ${clientName}`);
@@ -76,7 +78,7 @@ export class MCPClientManager {
         // Cache resources, if supported
         try {
             const resources = await client.listResources();
-            resources.forEach(resourceUri => {
+            resources.forEach((resourceUri) => {
                 this.resourceToClientMap.set(resourceUri, client);
             });
             logger.debug(`Cached resources for client: ${clientName}`);
@@ -287,9 +289,12 @@ export class MCPClientManager {
         for (const [name, client] of this.clients.entries()) {
             if (client.disconnect) {
                 disconnectPromises.push(
-                    client.disconnect()
+                    client
+                        .disconnect()
                         .then(() => logger.info(`Disconnected client: ${name}`))
-                        .catch(error => logger.error(`Failed to disconnect client '${name}': ${error}`))
+                        .catch((error) =>
+                            logger.error(`Failed to disconnect client '${name}': ${error}`)
+                        )
                 );
             }
         }
diff --git a/src/client/mcp-client.ts b/src/client/mcp-client.ts
index e33d60552..8f8812a3f 100644
--- a/src/client/mcp-client.ts
+++ b/src/client/mcp-client.ts
@@ -254,7 +254,7 @@ export class MCPClient implements IMCPClient {
         try {
             // Call listTools with parameters only
             const listToolResult = await this.client.listTools({});
-            logger.debug(`listTools result: ${JSON.stringify(listToolResult, null, 2)}`);
+            logger.silly(`listTools result: ${JSON.stringify(listToolResult, null, 2)}`);
 
             // Assume listToolResult.tools exists and iterate
             if (listToolResult && listToolResult.tools) {
@@ -283,6 +283,7 @@ export class MCPClient implements IMCPClient {
     /**
      * Get the list of prompts provided by this client
      * @returns Array of available prompt names
+     * TODO: Turn exception logs back into error and only call this based on capabilities of the server
      */
     async listPrompts(): Promise<string[]> {
         this.ensureConnected();
@@ -291,7 +292,9 @@ export class MCPClient implements IMCPClient {
             logger.debug(`listPrompts response: ${JSON.stringify(response, null, 2)}`);
             return response.prompts.map((p: any) => p.name);
         } catch (error) {
-            logger.error('Failed to list prompts from MCP server (optional feature), skipping:', error);
+            logger.debug(
+                `Failed to list prompts from MCP server (optional feature), skipping: ${JSON.stringify(error, null, 2)}`
+            );
             return [];
         }
     }
@@ -301,17 +304,21 @@ export class MCPClient implements IMCPClient {
      * @param name Name of the prompt
      * @param args Arguments for the prompt (optional)
      * @returns Prompt definition (structure depends on SDK)
+     * TODO: Turn exception logs back into error and only call this based on capabilities of the server
      */
     async getPrompt(name: string, args?: any): Promise<GetPromptResult> {
         this.ensureConnected();
         try {
             logger.debug(`Getting prompt '${name}' with args: ${JSON.stringify(args, null, 2)}`);
             // Pass params first, then options
-            const response = await this.client.getPrompt({ name, arguments: args }, { timeout: this.timeout });
+            const response = await this.client.getPrompt(
+                { name, arguments: args },
+                { timeout: this.timeout }
+            );
             logger.debug(`getPrompt '${name}' response: ${JSON.stringify(response, null, 2)}`);
             return response; // Return the full response object
         } catch (error: any) {
-            logger.error(`Failed to get prompt '${name}' from MCP server:`, error);
+            logger.debug(`Failed to get prompt '${name}' from MCP server: ${JSON.stringify(error, null, 2)}`);
             throw new Error(
                 `Error getting prompt '${name}': ${error instanceof Error ? error.message : String(error)}`
             );
@@ -321,6 +328,7 @@ export class MCPClient implements IMCPClient {
     /**
      * Get the list of resources provided by this client
      * @returns Array of available resource URIs
+     * TODO: Turn exception logs back into error and only call this based on capabilities of the server
      */
     async listResources(): Promise<string[]> {
         this.ensureConnected();
@@ -329,7 +337,9 @@ export class MCPClient implements IMCPClient {
             logger.debug(`listResources response: ${JSON.stringify(response, null, 2)}`);
             return response.resources.map((r: any) => r.uri);
         } catch (error) {
-            logger.error('Failed to list resources from MCP server (optional feature), skipping:', error);
+            logger.debug(
+                `Failed to list resources from MCP server (optional feature), skipping: ${JSON.stringify(error, null, 2)}`
+            );
             return [];
         }
     }
@@ -348,7 +358,7 @@ export class MCPClient implements IMCPClient {
             logger.debug(`readResource '${uri}' response: ${JSON.stringify(response, null, 2)}`);
             return response; // Return the full response object
         } catch (error: any) {
-            logger.error(`Failed to read resource '${uri}' from MCP server:`, error);
+            logger.debug(`Failed to read resource '${uri}' from MCP server: ${JSON.stringify(error, null, 2)}`);
             throw new Error(
                 `Error reading resource '${uri}': ${error instanceof Error ? error.message : String(error)}`
             );
diff --git a/src/config/loader.ts b/src/config/loader.ts
index 6b582d294..7b16d38e0 100644
--- a/src/config/loader.ts
+++ b/src/config/loader.ts
@@ -39,7 +39,7 @@ export async function loadConfigFile(configPath: string): Promise<AgentConfig> {
         const resolveFromPackageRoot = configPath === DEFAULT_CONFIG_PATH;
         const absolutePath = resolvePackagePath(configPath, resolveFromPackageRoot);
 
-        logger.debug(`Attempting to load config from: ${absolutePath}`);
+        logger.debug(`Loading saiki config from: ${absolutePath}`);
 
         // Read and parse the config file
         const fileContent = await fs.readFile(absolutePath, 'utf-8');
diff --git a/src/config/manager.ts b/src/config/manager.ts
new file mode 100644
index 000000000..35fa1e67e
--- /dev/null
+++ b/src/config/manager.ts
@@ -0,0 +1,119 @@
+import { z } from 'zod';
+import { logger } from '../utils/logger.js';
+import { llmConfigSchema } from './schemas.js';
+import type { AgentConfig, CLIConfigOverrides, LLMConfig, LLMProvenance } from './types.js';
+
+/**
+ * ConfigManager encapsulates merging file-based configuration, CLI overrides,
+ * default values, validation, and provenance tracking for all configuration
+ * sections (e.g., LLM settings, MCP servers, and more).
+ *
+ * Provenance records the origin of each configuration field—whether the value
+ * came from the configuration file, a CLI override, or a default.
+ */
+export class ConfigManager {
+    private resolved: AgentConfig;
+    private provenance: { llm: LLMProvenance };
+
+    constructor(fileConfig: AgentConfig) {
+        this.resolved = structuredClone(fileConfig);
+        this.provenance = { llm: { provider: 'file', model: 'file', router: 'default' } };
+        this.applyDefaults();
+    }
+
+    private applyDefaults() {
+        if (!this.resolved.llm.router) {
+            this.resolved.llm.router = 'vercel';
+            this.provenance.llm.router = 'default';
+        }
+    }
+
+    /** Apply CLI overrides and record provenance */
+    overrideCLI(cliArgs: CLIConfigOverrides) {
+        logger.debug('Applying CLI overrides to LLM config');
+        if (cliArgs.provider) {
+            this.resolved.llm.provider = cliArgs.provider;
+            this.provenance.llm.provider = 'cli';
+        }
+        if (cliArgs.model) {
+            this.resolved.llm.model = cliArgs.model;
+            this.provenance.llm.model = 'cli';
+        }
+        if (cliArgs.router) {
+            this.resolved.llm.router = cliArgs.router;
+            this.provenance.llm.router = 'cli';
+        }
+        if (cliArgs.apiKey) {
+            this.resolved.llm.apiKey = cliArgs.apiKey;
+            // Optionally track provenance for apiKey if desired
+        }
+        return this;
+    }
+
+    /** Returns the fully resolved AgentConfig */
+    getConfig(): AgentConfig {
+        return this.resolved;
+    }
+
+    /** Returns the provenance map for LLM settings */
+    getProvenance(): { llm: LLMProvenance } {
+        return this.provenance;
+    }
+
+    /** Pretty-print the resolved config and provenance */
+    print(): void {
+        logger.info('Resolved configuration:');
+        logger.info(JSON.stringify(this.resolved, null, 2));
+        logger.info('Configuration sources:');
+        for (const [field, src] of Object.entries(this.provenance.llm)) {
+            logger.info(`  • ${field}: ${src}`);
+        }
+    }
+
+    /**
+     * Validate the current resolved config. Throws Error with both schema issues and provenance.
+     * Delegates to helper methods for each section of the config.
+     */
+    validate(): void {
+        try {
+            this.validateMcpServers();
+            this.validateLlm();
+            logger.debug('LLM config validation successful', 'green');
+        } catch (err) {
+            // On validation failure, dump resolved config and provenance for debugging
+            this.print();
+            throw err;
+        }
+    }
+
+    /**
+     * Validates the MCP servers configuration
+     */
+    private validateMcpServers(): void {
+        if (!this.resolved.mcpServers || Object.keys(this.resolved.mcpServers).length === 0) {
+            throw new Error('No MCP server configurations provided in the resolved config.');
+        }
+    }
+
+    /**
+     * Validates the LLM configuration
+     */
+    private validateLlm(): void {
+        if (!this.resolved.llm) {
+            throw new Error('LLM configuration is missing in the resolved config.');
+        }
+        try {
+            llmConfigSchema.parse(this.resolved.llm);
+        } catch (err) {
+            if (err instanceof z.ZodError) {
+                const issues = err.errors.map((e) => {
+                    const p = e.path.join('.') || 'config';
+                    return `  • ${p}: ${e.message}`;
+                });
+                throw new Error(`Invalid LLM configuration:\n${issues.join('\n')}`);
+            }
+            const msg = err instanceof Error ? err.message : String(err);
+            throw new Error(`Unexpected error during LLM config validation: ${msg}`);
+        }
+    }
+}
diff --git a/src/config/schemas.ts b/src/config/schemas.ts
new file mode 100644
index 000000000..d6c9c5311
--- /dev/null
+++ b/src/config/schemas.ts
@@ -0,0 +1,56 @@
+import { z } from 'zod';
+import type { LLMConfig, AgentConfig, ContributorConfig, SystemPromptConfig } from './types.js';
+import { SchemaFromInterface } from '../utils/zod.js';
+import {
+    getSupportedProviders,
+    getSupportedModels,
+    isValidProviderModel,
+} from '../ai/llm/registry.js';
+
+export const contributorConfigSchema = z.object({
+    id: z.string(),
+    type: z.union([z.literal('static'), z.literal('dynamic')]),
+    priority: z.number(),
+    enabled: z.boolean().optional(),
+    content: z.string().optional(),
+    source: z.string().optional(),
+}) satisfies SchemaFromInterface<ContributorConfig>;
+
+export const systemPromptConfigSchema = z.object({
+    contributors: z.array(contributorConfigSchema),
+});
+
+// NOTE: We cannot use SchemaFromInterface here because the 'systemPrompt' property is a union type (string | SystemPromptConfig),
+// which is not supported by the mapped type utility. This schema must be maintained separately from the LLMConfig interface.
+export const llmConfigSchema = z
+    .object({
+        provider: z.string(),
+        model: z.string(),
+        systemPrompt: z.union([z.string(), systemPromptConfigSchema]),
+        apiKey: z.string().optional(),
+        maxIterations: z.number().optional(),
+        providerOptions: z.record(z.any()).optional(),
+        router: z.any().optional(),
+    })
+    .superRefine((data, ctx) => {
+        // 1. Provider must be one of the supported list
+        const supportedProviders = getSupportedProviders();
+        if (!supportedProviders.includes(data.provider.toLowerCase())) {
+            ctx.addIssue({
+                code: z.ZodIssueCode.custom,
+                path: ['provider'],
+                message: `Provider '${data.provider}' is not supported. Supported: ${supportedProviders.join(', ')}`,
+            });
+        }
+        // 2. Model must be valid for that provider
+        const supportedModels = getSupportedModels(data.provider);
+        if (!isValidProviderModel(data.provider, data.model)) {
+            ctx.addIssue({
+                code: z.ZodIssueCode.custom,
+                path: ['model'],
+                message: `Model '${data.model}' is not supported for provider '${data.provider}'. Supported: ${supportedModels.join(', ')}`,
+            });
+        }
+    });
+
+// You can add more schemas for AgentConfig, etc., as needed.
diff --git a/src/config/types.ts b/src/config/types.ts
index 854c7c15f..c4ed2ba24 100644
--- a/src/config/types.ts
+++ b/src/config/types.ts
@@ -2,6 +2,8 @@
  * Type definitions for application configuration
  */
 
+import { LLMRouter } from '../ai/llm/types.js';
+
 /**
  * Configuration for stdio-based MCP server connections
  * - type: Must be 'stdio'
@@ -61,6 +63,7 @@ export type LLMConfig = {
     /** Maximum number of tool-iteration steps (for in-built and Vercel providers) */
     maxIterations?: number;
     providerOptions?: Record<string, any>;
+    router?: LLMRouter; // Optional router field
 };
 
 /**
@@ -71,3 +74,27 @@ export type AgentConfig = {
     llm: LLMConfig;
     [key: string]: any; // Allow for future extensions
 };
+
+/**
+ * CLI config override type for allowed fields
+ */
+export type CLIConfigOverrides = Partial<{
+    model: string;
+    provider: string;
+    router: LLMRouter;
+    apiKey: string;
+}>;
+
+/**
+ * Possible sources for configuration field overrides
+ */
+export type Source = 'file' | 'cli' | 'default';
+
+/**
+ * Provenance for configuration fields: tracks where a value originated from
+ */
+export type LLMProvenance = {
+    provider: Source;
+    model: Source;
+    router: Source;
+};
diff --git a/src/utils/logger.ts b/src/utils/logger.ts
index 758ca5f28..419937fc7 100644
--- a/src/utils/logger.ts
+++ b/src/utils/logger.ts
@@ -57,6 +57,24 @@ const consoleFormat = winston.format.printf(({ level, message, timestamp, color
     return `${chalk.dim(timestamp)} ${colorize(level.toUpperCase())}: ${formattedMessage}`;
 });
 
+/**
+ * Logic to redact sensitive information from logs.
+ * This is useful for preventing sensitive information from being logged in production.
+ * On by default, we can set the environment variable REDACT_SECRETS to false to disable this behavior.
+ */
+const SHOULD_REDACT = process.env.REDACT_SECRETS !== 'false';
+const SENSITIVE_KEYS = ['apiKey', 'password', 'secret', 'token'];
+const MASK_REGEX = new RegExp(
+    `(${SENSITIVE_KEYS.join('|')})(["']?\\s*[:=]\\s*)(["'])?.*?\\3`,
+    'gi'
+);
+const maskFormat = winston.format((info) => {
+    if (SHOULD_REDACT && typeof info.message === 'string') {
+        info.message = info.message.replace(MASK_REGEX, '$1$2$3[REDACTED]$3');
+    }
+    return info;
+});
+
 export interface LoggerOptions {
     level?: string;
     silent?: boolean;
@@ -86,6 +104,7 @@ export class Logger {
             silent: options.silent || false,
             format: winston.format.combine(
                 winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss' }),
+                maskFormat(),
                 winston.format.errors({ stack: true }),
                 winston.format.splat(),
                 winston.format.json()
@@ -94,6 +113,7 @@ export class Logger {
                 new winston.transports.Console({
                     format: winston.format.combine(
                         winston.format.timestamp({ format: 'HH:mm:ss' }),
+                        maskFormat(),
                         consoleFormat
                     ),
                 }),
diff --git a/src/utils/options.ts b/src/utils/options.ts
new file mode 100644
index 000000000..77def0d0e
--- /dev/null
+++ b/src/utils/options.ts
@@ -0,0 +1,77 @@
+import { z } from 'zod';
+import { logger } from './logger.js';
+import { getSupportedProviders, getSupportedModels } from '../ai/llm/registry.js';
+
+/**
+ * Validates the command-line options.
+ * @param opts - The command-line options object from commander.
+ * @throws {z.ZodError} If validation fails.
+ */
+export function validateCliOptions(opts: any): void {
+    logger.debug('Validating command-line options', 'cyanBright');
+
+    const supportedProviders = getSupportedProviders().map((p) => p.toLowerCase());
+
+    // Base schema for primitive shape
+    const cliOptionShape = z.object({
+        configFile: z.string().nonempty('Config file path must not be empty'),
+        strict: z.boolean().optional().default(false),
+        verbose: z.boolean().optional().default(true),
+        mode: z.enum(['cli', 'web'], {
+            errorMap: () => ({ message: 'Mode must be either "cli" or "web"' }),
+        }),
+        webPort: z.string().refine(
+            (val) => {
+                const port = parseInt(val, 10);
+                return !isNaN(port) && port > 0 && port <= 65535;
+            },
+            { message: 'Web port must be a number between 1 and 65535' }
+        ),
+        provider: z.string().optional(),
+        model: z.string().optional(),
+        router: z.enum(['vercel', 'in-built']).optional(),
+    });
+
+    // Basic semantic validation
+    const cliOptionSchema = cliOptionShape
+        // 1) provider must be one of the supported set if provided
+        .refine(
+            (data) => !data.provider || supportedProviders.includes(data.provider.toLowerCase()),
+            {
+                path: ['provider'],
+                message: `Provider must be one of: ${supportedProviders.join(', ')}`,
+            }
+        );
+
+    // Execute validation
+    cliOptionSchema.parse({
+        configFile: opts.configFile,
+        strict: opts.strict,
+        verbose: opts.verbose,
+        mode: opts.mode.toLowerCase(),
+        webPort: opts.webPort,
+        provider: opts.provider,
+        model: opts.model,
+        router: opts.router,
+    });
+
+    logger.debug('Command-line options validated successfully', 'green');
+}
+
+export function handleCliOptionsError(error: unknown): never {
+    if (error instanceof z.ZodError) {
+        logger.error('Invalid command-line options detected:');
+        error.errors.forEach((err) => {
+            const fieldName = err.path.join('.') || 'Unknown Option';
+            logger.error(`- Option '${fieldName}': ${err.message}`);
+        });
+        logger.error(
+            'Please check your command-line arguments or run with --help for usage details.'
+        );
+    } else {
+        logger.error(
+            `Validation error: ${error instanceof Error ? error.message : JSON.stringify(error)}`
+        );
+    }
+    process.exit(1);
+}
diff --git a/src/utils/service-initializer.ts b/src/utils/service-initializer.ts
index c1d435dd0..b3ef927af 100644
--- a/src/utils/service-initializer.ts
+++ b/src/utils/service-initializer.ts
@@ -36,6 +36,9 @@ import { MessageManager } from '../ai/llm/messages/manager.js';
 import { createMessageManager } from '../ai/llm/messages/factory.js';
 import { createToolConfirmationProvider } from '../client/tool-confirmation/factory.js';
 import { loadContributors } from '../ai/systemPrompt/loader.js';
+import { loadConfigFile } from '../config/loader.js';
+import { ConfigManager } from '../config/manager.js';
+import type { CLIConfigOverrides } from '../config/types.js';
 
 /**
  * Type for the core agent services returned by initializeServices
@@ -45,6 +48,7 @@ export type AgentServices = {
     llmService: ILLMService;
     agentEventBus: EventEmitter;
     messageManager: MessageManager;
+    configManager: ConfigManager;
 };
 
 /**
@@ -69,67 +73,64 @@ export type InitializeServicesOptions = {
     clientManager?: MCPClientManager; // Inject a custom or mock MCPClientManager
     llmService?: ILLMService; // Inject a custom or mock LLMService
     agentEventBus?: EventEmitter; // Inject a custom or mock EventEmitter
-    llmRouter?: LLMRouter; // Route LLM calls via Vercel (default) or use in-built
     messageManager?: MessageManager; // Inject a custom or mock MessageManager
     // Add more overrides as needed
     // configOverride?: Partial<AgentConfig>; // (optional) for field-level config overrides
 };
 
+// High-level factory to load, validate, and wire up all agent services in one call
 /**
- * Initialize services and clients based on the provided configuration and optional overrides.
- * This is the central point for creating all services and dependencies.
- * @param config Agent configuration including MCP servers and LLM settings
- * @param options Optional overrides for testability, context, or advanced configuration
- * @returns All the initialized services/dependencies necessary to run saiki
+ * Loads and validates configuration and initializes all agent services as a single unit.
+ * @param configPath Path to the agent config file
+ * @param cliArgs Overrides from the CLI
+ * @param overrides Optional service overrides for testing or advanced scenarios
+ * @returns All the initialized services and the config manager
  */
-export async function initializeServices(
-    config: AgentConfig,
-    options?: InitializeServicesOptions
+export async function createAgentServices(
+    configPath: string,
+    cliArgs: CLIConfigOverrides,
+    overrides?: InitializeServicesOptions
 ): Promise<AgentServices> {
-    /**
-     * 1. Create or use the shared event bus (allows override for tests/mocks)
-     */
-    const agentEventBus = options?.agentEventBus ?? new EventEmitter();
+    // 1. Load raw configuration from file
+    const rawConfig = await loadConfigFile(configPath);
+
+    // 2. Initialize config manager, apply CLI config level overrides and validate
+    const configManager = new ConfigManager(rawConfig).overrideCLI(cliArgs);
+    configManager.validate();
+    const config = configManager.getConfig();
+
+    // 3. Initialize shared event bus
+    const agentEventBus = overrides?.agentEventBus ?? new EventEmitter();
     logger.debug('Agent event bus initialized');
 
-    /**
-     * 2. Initialize or use the client manager (allows override for tests/mocks)
-     *    - Selects the appropriate ToolConfirmationProvider based on runMode (cli/web/etc) using the factory.
-     */
-    const connectionMode = options?.connectionMode ?? 'lenient';
-    const runMode = options?.runMode ?? 'cli';
+    // 4. Initialize client manager
+    const connectionMode = overrides?.connectionMode ?? 'lenient';
+    const runMode = overrides?.runMode ?? 'cli';
     const confirmationProvider = createToolConfirmationProvider(runMode);
-    const clientManager = options?.clientManager ?? new MCPClientManager(confirmationProvider);
+    const clientManager = overrides?.clientManager ?? new MCPClientManager(confirmationProvider);
     await clientManager.initializeFromConfig(config.mcpServers, connectionMode);
-    if (!options?.clientManager) {
-        logger.debug('Client manager and MCP servers initialized');
-    } else {
-        logger.debug('Client manager and MCP servers initialized via options override');
-    }
+    logger.debug(
+        overrides?.clientManager
+            ? 'Client manager and MCP servers initialized via override'
+            : 'Client manager and MCP servers initialized'
+    );
 
-    /**
-     * 3. Initialize or use the MessageManager (allows override for tests/mocks)
-     *    - Uses llmRouter from options to select the correct message formatting/tokenization backend
-     *    - 'vercel' = Vercel-style message formatting (default), 'default' = in-built provider-specific formatting
-     */
-    const router: LLMRouter = options?.llmRouter ?? 'vercel';
+    // 5. Initialize message manager
+    const router: LLMRouter = config.llm.router ?? 'vercel';
     const contributors = loadContributors(config.llm.systemPrompt);
     const messageManager =
-        options?.messageManager ?? createMessageManager(config.llm, router, contributors);
+        overrides?.messageManager ?? createMessageManager(config.llm, router, contributors);
 
-    /**
-     * 4. Initialize or use the LLMService (allows override for tests/mocks)
-     *    - Use llmRouter from options to select LLM routing backend
-     *    - 'vercel' = route via Vercel LLM service (default), 'default' = use in-built LLM services
-     */
+    // 6. Initialize LLM service
     const llmService =
-        options?.llmService ??
+        overrides?.llmService ??
         createLLMService(config.llm, router, clientManager, agentEventBus, messageManager);
-    if (!options?.llmService) {
-        logger.debug(`LLM service initialized using router: ${router}`);
-    } else {
-        logger.debug('LLM service provided via options override');
-    }
+    logger.debug(
+        overrides?.llmService
+            ? 'LLM service provided via override'
+            : `LLM service initialized using router: ${router}`
+    );
 
-    return { clientManager, llmService, agentEventBus, messageManager };
+    // 7. Return the full service graph, including the ConfigManager
+    return { clientManager, llmService, agentEventBus, messageManager, configManager };
 }
diff --git a/src/utils/zod.ts b/src/utils/zod.ts
new file mode 100644
index 000000000..5d287c623
--- /dev/null
+++ b/src/utils/zod.ts
@@ -0,0 +1,10 @@
+import { ZodObject, ZodType } from 'zod';
+
+// New utility type for compile-time checking of Zod schemas against TypeScript interfaces
+// This enables us to tightly couple zod schemas with the typescript interfaces we use in our codebase.
+// This is useful for ensuring that the zod schemas are always in sync with the interfaces, and for providing
+// better type safety when using the schemas.
+// We need to ensure that we update usages of this type whenever we update the interfaces it is based on.
+export type SchemaFromInterface<T> = ZodObject<{
+    [K in keyof Partial<T>]: K extends keyof T ? ZodType<T[K]> : never;
+}>;


Reviews:
user: login: shaunak99
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-02T05:57:12Z
user: login: shaunak99
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-02T06:02:17Z
user: login: rahulkarajgikar
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-02T07:39:18Z

Review comments:
diff_hunk: @@ -291,7 +291,10 @@ export class MCPClient implements IMCPClient {
             logger.debug(`listPrompts response: ${JSON.stringify(response, null, 2)}`);
             return response.prompts.map((p: any) => p.name);
         } catch (error) {
-            logger.error('Failed to list prompts from MCP server (optional feature), skipping:', error);
+            logger.error(
+                'Failed to list prompts from MCP server (optional feature), skipping:',
path: src/client/mcp-client.ts
user: login: shaunak99
body: Can we make this something not error? It throws errors on regular startups when the server doesn't support lists or tools. Shouldn't through error, maybe debug?
created_at: 2025-05-02T05:57:12Z
author_association: COLLABORATOR
diff_hunk: @@ -329,7 +335,10 @@ export class MCPClient implements IMCPClient {
             logger.debug(`listResources response: ${JSON.stringify(response, null, 2)}`);
             return response.resources.map((r: any) => r.uri);
         } catch (error) {
-            logger.error('Failed to list resources from MCP server (optional feature), skipping:', error);
+            logger.error(
+                'Failed to list resources from MCP server (optional feature), skipping:',
path: src/client/mcp-client.ts
user: login: shaunak99
body: same as above
created_at: 2025-05-02T05:57:15Z
author_association: COLLABORATOR
diff_hunk: @@ -291,7 +291,10 @@ export class MCPClient implements IMCPClient {
             logger.debug(`listPrompts response: ${JSON.stringify(response, null, 2)}`);
             return response.prompts.map((p: any) => p.name);
         } catch (error) {
-            logger.error('Failed to list prompts from MCP server (optional feature), skipping:', error);
+            logger.error(
+                'Failed to list prompts from MCP server (optional feature), skipping:',
path: src/client/mcp-client.ts
user: login: rahulkarajgikar
body: yeah will make it debug for now

we should ideally not call this function at all if the server doesn't support this capability.

added that as a todo comment, will make a github issue out of it
created_at: 2025-05-02T07:39:18Z
author_association: COLLABORATOR

PR comments:
user: login: rahulkarajgikar
created_at: 2025-05-01T04:22:55Z
body: @CodiumAI-Agent /review
author_association: COLLABORATOR
user: login: QodoAI-Agent
created_at: 2025-05-01T04:24:02Z
body: ## PR Reviewer Guide 🔍

#### (Review updated until commit https://github.com/truffle-ai/saiki/commit/97f6c9189689d483a0b031a7d3dad47147419892)


Here are some key observations to aid the review process:

<table>
<tr><td>

**🎫 Ticket compliance analysis 🔶**



**[84](https://github.com/truffle-ai/saiki/issues/84) - Partially compliant**

Compliant requirements:

- Support CLI option `-m, --model <model>` to specify model at startup.
- Infer and set provider dynamically based on the provided model.
- Override model and provider from configuration file with CLI inputs.
- Provide user-friendly error and list supported models when an invalid model is given.
- Ensure environment variables for API keys are correctly applied based on the model provider.

Non-compliant requirements:

- Switching the model dynamically during runtime (after startup) is not yet supported.



</td></tr>
<tr><td>⏱️&nbsp;<strong>Estimated effort to review</strong>: 4 🔵🔵🔵🔵⚪</td></tr>
<tr><td>🧪&nbsp;<strong>No relevant tests</strong></td></tr>
<tr><td>🔒&nbsp;<strong>No security concerns identified</strong></td></tr>
<tr><td>⚡&nbsp;<strong>Recommended focus areas for review</strong><br><br>

<details><summary><a href='https://github.com/truffle-ai/saiki/pull/85/files#diff-8571db5cff3b2743c084a212c967aa38286b58bffa76e7d8320e4d82c7295d95R30-R44'><strong>Validation Gap</strong></a>

The CLI validation schema does not enforce that the provided model is valid for the selected provider, which may lead to runtime errors instead of early rejections.
</summary>

```typescript
    provider: z.string().optional(),
    model: z.string().optional(),
    router: z.enum(['vercel', 'in-built']).optional(),
});

// Basic semantic validation
const cliOptionSchema = cliOptionShape
    // 1) provider must be one of the supported set if provided
    .refine(
        (data) => !data.provider || supportedProviders.includes(data.provider.toLowerCase()),
        {
            path: ['provider'],
            message: `Provider must be one of: ${supportedProviders.join(', ')}`,
        }
    );
```

</details>

<details><summary><a href='https://github.com/truffle-ai/saiki/pull/85/files#diff-0e169bdcf1e872b1ce03646c2dd80a587a9257a63a24e641592106346a6e2b66R19-R27'><strong>Provenance Mismatch</strong></a>

Provenance for default router is labeled as 'default' while the router enum uses 'vercel' or 'in-built', which may confuse the source tracking of routing settings.
</summary>

```typescript
    this.resolved = structuredClone(fileConfig);
    this.provenance = { llm: { provider: 'file', model: 'file', router: 'default' } };
    this.applyDefaults();
}

private applyDefaults() {
    if (!this.resolved.llm.router) {
        this.resolved.llm.router = 'vercel';
        this.provenance.llm.router = 'default';
```

</details>

<details><summary><a href='https://github.com/truffle-ai/saiki/pull/85/files#diff-b531b96dd0079473aedc9e22fe110d7e1f5bac39a62a788614227b710811fe94R128-R135'><strong>Unused Return Value</strong></a>

`createAgentServices` returns a `configManager` instance that is not used in `startAgent`, consider using it or removing it from the return value.
</summary>

```typescript
async function startAgent() {
    // Use createAgentServices to load, validate config and initialize all agent services
    const cliArgs = { model: options.model, provider: options.provider, router: options.router, apiKey: options.apiKey };
    let services;
    try {
        services = await createAgentServices(normalizedConfigPath, cliArgs, {
            connectionMode,
            runMode,
```

</details>

</td></tr>
</table>

author_association: NONE
user: login: rahulkarajgikar
created_at: 2025-05-01T04:24:51Z
body: @CodiumAI-Agent /describe
author_association: COLLABORATOR
user: login: QodoAI-Agent
created_at: 2025-05-01T04:25:36Z
body: ## Title

[WIP] Configuration changes

___
### **User description**
- feature: adding model/provider/router arguments to CLI commands
 - feature: added a utility to compare zod schemas with typescript interfaces that they represent - to keep schemas and interface in sync `zod.ts`. useful for codebase cleanliness.
 - centralized all supported LLM models and token counts to one file llm/registry.ts. (just one code change to add a new model/provider now)
 - cleaned up starting file `index.ts`
 - moved agent config validation to a separate file config/validator.ts


___

### **PR Type**
Enhancement


___

### **Description**
Introduce central LLM registry with model metadata
Replace old token utils with registry getMaxTokensForModel
Add resolveConfiguration for CLI overrides
Validate CLI and agent config via Zod schemas


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table>
<tr>
  <td><strong>index.ts</strong><dd><code>Integrate Zod validation and config resolution</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-b531b96dd0079473aedc9e22fe110d7e1f5bac39a62a788614227b710811fe94">+55/-82</a>&nbsp; </td>

</tr>

<tr>
  <td><strong>factory.ts</strong><dd><code>Use registry tokens and add tokenizer error handling</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-7f68d55fb314b19c26b7230af3a644519fc211667f6708f7af9654f3792e139f">+22/-8</a>&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>registry.ts</strong><dd><code>Add central LLM registry and helper functions</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-4b9d1bc9b062dffd4f35c4da6735bdeb7b41e94bd8896fd16dbcbdeaf2718909">+117/-0</a>&nbsp; </td>

</tr>

<tr>
  <td><strong>anthropic.ts</strong><dd><code>Replace getMaxTokens with registry lookup</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-c706ef55ab241c31847538ad6cd9e1ebb9dd23f72ea43caf3e89a3f7e4bbbcb9">+3/-2</a>&nbsp; &nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>openai.ts</strong><dd><code>Replace getMaxTokens with registry lookup</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-9dee06fb082b04a5d3ce91ba17407b88744da968b1e925e5b2611dddaf7ec6f3">+3/-2</a>&nbsp; &nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>vercel.ts</strong><dd><code>Replace getMaxTokens with registry lookup</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-ff4bf25995d02ca7661923edb1ec79bf562ae248874f7c0bc69e585d23787d2a">+4/-3</a>&nbsp; &nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>service-initializer.ts</strong><dd><code>Use resolved config router in initialization</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-fb4d087b794ea9eef96c6d51f6add201dd5f45029e5d154469445b63ca42567e">+5/-8</a>&nbsp; &nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>zod.ts</strong><dd><code>Add SchemaFromInterface Zod utility type</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-a94feb9526c2f2536e8edd3150f2ba0a88c4e29a14f69c199d54682ab49cf38e">+10/-0</a>&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>1 files</summary><table>
<tr>
  <td><strong>utils.ts</strong><dd><code>Remove deprecated tokenizer utils</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-065874d791f35f4545ec6a9051e0224ac14ade99cef3b9f93c906e2e3b430f8c">+0/-91</a>&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>3 files</summary><table>
<tr>
  <td><strong>resolver.ts</strong><dd><code>Add CLI override configuration resolver</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-3adaa8ad7f92d8f6dea09fbc7bb7e1e6e6c2f795aafbc26bf4df2667d3225448">+39/-0</a>&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>schemas.ts</strong><dd><code>Define Zod schemas for config types</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-201d08e10151aeecdf70e8125e06cc2fc1be5de186dab198ea4a53ae446323bc">+43/-0</a>&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>types.ts</strong><dd><code>Extend config types with CLI overrides</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-949e6a9a52e37102f2a5a698afcaa03c668a95bea1a35f0efb0ef0cb77eb5ce3">+12/-0</a>&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table>
<tr>
  <td><strong>validator.ts</strong><dd><code>Implement Zod-based option and config validation</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-15cd5a9140b94fc4277f88759c667e2b72022e911cc14b089643327fd288ed7b">+80/-0</a>&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Dependencies</strong></td><td><details><summary>1 files</summary><table>
<tr>
  <td><strong>package.json</strong><dd><code>Update zod and add ts-to-zod dependency</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
  <td><a href="https://github.com/truffle-ai/saiki/pull/85/files#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519">+3/-2</a>&nbsp; &nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> <details> <summary>  Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details>
author_association: NONE
user: login: rahulkarajgikar
created_at: 2025-05-01T04:37:24Z
body: @CodiumAI-Agent /improve
author_association: COLLABORATOR
user: login: QodoAI-Agent
created_at: 2025-05-01T04:39:01Z
body: ## PR Code Suggestions ✨
<!-- 7d0153e -->

Latest suggestions up to 7d0153e
<table><thead><tr><td><strong>Category</strong></td><td align=left><strong>Suggestion&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </strong></td><td align=center><strong>Impact</strong></td></tr><tbody><tr><td rowspan=1>Possible issue</td>
<td>



<details><summary>Include error stack trace</summary>

___


**Log the full error stack instead of only splitting the message to improve debugging. <br>Replace the manual message split with a single call to log the stack trace.**

[app/index.ts [120-127]](https://github.com/truffle-ai/saiki/pull/85/files#diff-b531b96dd0079473aedc9e22fe110d7e1f5bac39a62a788614227b710811fe94R120-R127)

```diff
 } catch (err) {
     if (err instanceof Error) {
-        err.message.split('\n').forEach((line) => logger.error(line));
+        logger.error(err.stack ?? err.message);
     } else {
         logger.error('Unexpected error during startup:', err);
     }
     process.exit(1);
 }
```
<details><summary>Suggestion importance[1-10]: 5</summary>

__

Why: The suggestion correctly replaces manual message splitting with logging the full error stack, improving debugging without altering core logic.


</details></details></td><td align=center>Low

</td></tr></tr></tbody></table>

___

#### Previous suggestions
<details><summary>Suggestions up to commit 1fff7b7</summary>
<br><table><thead><tr><td><strong>Category</strong></td><td align=left><strong>Suggestion&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </strong></td><td align=center><strong>Impact</strong></td></tr><tbody><tr><td rowspan=3>General</td>
<td>



<details><summary>Handle registry lookup errors</summary>

___


**Wrap the token-limit lookup in a try/catch to log and rethrow registry errors with <br>context. This avoids uncaught exceptions if the provider or model is missing. Use <br>the error message to indicate which lookup failed.**

[src/ai/llm/messages/factory.ts [33-38]](https://github.com/truffle-ai/saiki/pull/85/files#diff-7f68d55fb314b19c26b7230af3a644519fc211667f6708f7af9654f3792e139fR33-R38)

```diff
 const provider = config.provider.toLowerCase();
 const model = config.model.toLowerCase();
-const registryMaxTokens = getMaxTokensForModel(provider, model);
-maxTokens = Math.floor(registryMaxTokens! * 0.9);
+let registryMaxTokens: number;
+try {
+    registryMaxTokens = getMaxTokensForModel(provider, model);
+} catch (err) {
+    logger.error(`Failed to retrieve max tokens for ${provider}/${model}: ${err}`);
+    throw err;
+}
+maxTokens = Math.floor(registryMaxTokens * 0.9);
```
<details><summary>Suggestion importance[1-10]: 7</summary>

__

Why: Wrapping `getMaxTokensForModel` in a `try/catch` prevents uncaught exceptions for missing provider/model and provides clear logging context, improving robustness.


</details></details></td><td align=center>Medium

</td></tr><tr><td>



<details><summary>Preserve original Zod errors</summary>

___


**Avoid creating a new <code>ZodError</code> which drops original context; rethrow the caught error <br>directly for complete stack and issue information. This preserves paths and <br>metadata.**

[src/config/validator.ts [63-74]](https://github.com/truffle-ai/saiki/pull/85/files#diff-15cd5a9140b94fc4277f88759c667e2b72022e911cc14b089643327fd288ed7bR63-R74)

```diff
 try {
     llmConfigSchema.parse(config.llm);
 } catch (error) {
-     // Re-throw Zod errors to be caught by the main try/catch block
+     // Re-throw Zod errors for structured handling
      if (error instanceof z.ZodError) {
-        // Construct a more informative error message from Zod issues
-        const errorSummary = error.errors.map(e => e.message).join('; ');
-        throw new z.ZodError(error.errors); // Re-throw the original Zod error for structured handling
+        throw error;
      }
      // Throw other unexpected errors during validation
      throw new Error(`Unexpected error during LLM config validation: ${error}`);
 }
```
<details><summary>Suggestion importance[1-10]: 6</summary>

__

Why: Rethrowing the caught `ZodError` preserves full error paths and metadata instead of creating a new error, improving debuggability with minimal changes.


</details></details></td><td align=center>Low

</td></tr><tr><td>



<details><summary>Fallback deep clone method</summary>

___


**Provide a fallback clone implementation when <code>structuredClone</code> isn't available (e.g., <br>older Node versions). This ensures safe deep copying in all supported environments.**

[src/config/resolver.ts [16]](https://github.com/truffle-ai/saiki/pull/85/files#diff-3adaa8ad7f92d8f6dea09fbc7bb7e1e6e6c2f795aafbc26bf4df2667d3225448R16-R16)

```diff
-const resolved: AgentConfig = structuredClone(configFile);
+const resolved: AgentConfig = typeof structuredClone === 'function'
+  ? structuredClone(configFile)
+  : JSON.parse(JSON.stringify(configFile));
```
<details><summary>Suggestion importance[1-10]: 3</summary>

__

Why: Providing a `JSON.parse(JSON.stringify(...))` fallback for `structuredClone` covers older runtimes, but most supported Node versions include `structuredClone`, making this a low-impact improvement.


</details></details></td><td align=center>Low

</td></tr><tr><td rowspan=1>Possible issue</td>
<td>



<details><summary>Add default system prompt</summary>

___


**Add a fallback default system prompt if none is provided so downstream logic always <br>has a prompt. This ensures the agent won’t fail when the config file omits a system <br>prompt. Import or define a <code>DEFAULT_SYSTEM_PROMPT</code> constant for reuse.**

[src/config/resolver.ts [16-34]](https://github.com/truffle-ai/saiki/pull/85/files#diff-3adaa8ad7f92d8f6dea09fbc7bb7e1e6e6c2f795aafbc26bf4df2667d3225448R16-R34)

```diff
 const resolved: AgentConfig = structuredClone(configFile);
 // Apply CLI arguments for allowed options (if provided)
 if (cliArgs.model) {
     logger.debug(`CLI resolved model: ${cliArgs.model}`);
     resolved.llm.model = cliArgs.model;
 }
 if (cliArgs.provider) {
     logger.debug(`CLI resolved provider: ${cliArgs.provider}`);
     resolved.llm.provider = cliArgs.provider;
 }
 if (cliArgs.router) {
     logger.debug(`CLI resolved router: ${cliArgs.router}`);
     resolved.llm.router = cliArgs.router;
 }
 // Ensure defaults if necessary (lowest priority)
 resolved.llm.router = resolved.llm.router ?? 'vercel';
+// Fallback default system prompt
+if (!resolved.llm.systemPrompt) {
+    logger.info('No system prompt found, applying default', 'yellow');
+    resolved.llm.systemPrompt = DEFAULT_SYSTEM_PROMPT;
+}
```
<details><summary>Suggestion importance[1-10]: 4</summary>

__

Why: Adding a fallback `systemPrompt` helps avoid runtime failures when the config omits it, but the PR may handle defaults elsewhere and this constant isn’t defined yet, so the change is helpful but not urgent or complete.


</details></details></td><td align=center>Low

</td></tr></tr></tbody></table>

</details>

author_association: NONE
user: login: rahulkarajgikar
created_at: 2025-05-02T01:21:45Z
body: Testing manually:
### 1.  incorrect model name should throw exception and inform users of available models
```
npm run build && npm start -- -m claude-3-7-sonnet-20250219a
```
Descriptive error with supported models list for UX
```
17:51:28 ERROR: ERROR: Unrecognized model 'claude-3-7-sonnet-20250219a'. Could not infer provider.
17:51:28 ERROR: Supported models are:
gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-mini, o4-mini, o3, o3-mini, o1, claude-3-7-sonnet-20250219, claude-3-5-sonnet-20240620, claude-3-haiku-20240307, claude-3-opus-20240229, claude-3-sonnet-20240229, gemini-2.5-pro, gemini-2.0-flash, gemini-2.0-flash-lite, gemini-1.5-pro-latest, gemini-1.5-flash-latest
```


### 2. correct model name should work without router specified - openai/anthropic/google

2.a: ```npm run build && npm start -- -m gpt-4o-mini```

correct model config loaded - defaults to vercel:
```
17:54:28 INFO: Using model config: {
  "router": "vercel",
  "provider": "openai",
  "model": {
    "specificationVersion": "v1",
    "modelId": "gpt-4o-mini",
    "settings": {},
    "config": {
      "provider": "openai.chat",
      "compatibility": "strict"
    }
  },
  "configuredMaxTokens": 115200,
  "modelMaxTokens": 128000
}
```

responses work:

```
┌──────────── 🤖 AI Response ────────────┐
│                                        │
│   Hello! How can I assist you today?   │
│                                        │
└────────────────────────────────────────┘
```


2.b: ```npm run build && npm start -- -m claude-3-7-sonnet-20250219```

correct model config loaded - defaults to vercel:
```
17:53:09 INFO: Using model config: {
  "router": "vercel",
  "provider": "anthropic",
  "model": {
    "specificationVersion": "v1",
    "defaultObjectGenerationMode": "tool",
    "supportsImageUrls": true,
    "modelId": "claude-3-7-sonnet-20250219",
    "settings": {},
    "config": {
      "provider": "anthropic.messages",
      "baseURL": "https://api.anthropic.com/v1"
    }
  },
  "configuredMaxTokens": 180000,
  "modelMaxTokens": 200000
}
```

response works as well:
```
Just let me know what you need, and I'll do my best to assist you.
17:53:16 DEBUG: Step finished, step tool calls: []
17:53:16 DEBUG: Step finished, step tool results: []
┌─────────────────────────────── 🤖 AI Response ───────────────────────────────┐
│                                                                              │
│   Hello! I'm Saiki, an AI assistant with access to various tools that can    │
│   help you browse the web, manage files, and more. How can I assist you      │
│   today?                                                                     │
│                                                                              │
│   Is there something specific you'd like to know or a task you'd like help   │
│   with? I'm here to help with:                                               │
│                                                                              │
│   - Web browsing and information retrieval                                   │
│   - File management (reading, writing, organizing files)                     │
│   - Answering questions                                                      │
│   - And more!                                                                │
│                                                                              │
│   Just let me know what you need, and I'll do my best to assist you.         │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

```

2.c: ``` npm run build && npm start -- -m gemini-2.0-flash```

```
17:55:43 INFO: Using model config: {
  "router": "vercel",
  "provider": "google",
  "model": {
    "specificationVersion": "v1",
    "defaultObjectGenerationMode": "json",
    "supportsImageUrls": false,
    "modelId": "gemini-2.0-flash",
    "settings": {},
    "config": {
      "provider": "google.generative-ai",
      "baseURL": "https://generativelanguage.googleapis.com/v1beta"
    }
  },
  "configuredMaxTokens": 943718,
  "modelMaxTokens": 1048576
}
```

response works
```
┌─────── 🤖 AI Response ────────┐
│                               │
│   How can I help you today?   │
│                               │
│                               │
└───────────────────────────────┘
```

some issue with 2.5-pro in vercel

```
18:22:06 ERROR: Error in Vercel LLM service execution: models/gemini-2.5-pro is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.
18:22:06 ERROR: ❌ Error: models/gemini-2.5-pro is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.
```

we might need to update sdk version



3. Same as above with `in-built` router instead of vercel

3.a ```npm run build && npm start -- -m gpt-4o-mini -r in-built```

```
18:19:00 INFO: Using model config: {
  "router": "in-built",
  "provider": "openai",
  "model": "gpt-4o-mini",
  "configuredMaxTokens": 115200,
  "modelMaxTokens": 128000
}
```

response worked

3.b `npm run build && npm start -- -m claude-3-7-sonnet-20250219 -r in-built`

```
18:20:16 INFO: Using model config: {
  "router": "in-built",
  "provider": "anthropic",
  "model": "claude-3-7-sonnet-20250219",
  "configuredMaxTokens": 180000,
  "modelMaxTokens": 200000
}
```

response worked

3.c ` npm run build && npm start -- -m gemini-2.0-flash -r in-built`

google not supported because we don't have an in-built llmservice for it - command exits early

```
18:20:59 ERROR: Provider 'google' supported by registry but not configured for 'default' router message formatting.
18:20:59 ERROR: Unsupported LLM provider: google for router: in-built`
```
author_association: COLLABORATOR
user: login: rahulkarajgikar
created_at: 2025-05-02T02:45:42Z
body: @CodiumAI-Agent /review
author_association: COLLABORATOR
user: login: QodoAI-Agent
created_at: 2025-05-02T02:47:17Z
body: **[Persistent review](https://github.com/truffle-ai/saiki/pull/85#issuecomment-2844065841)** updated to latest commit https://github.com/truffle-ai/saiki/commit/5866c6af521e34a3a0d6777da54bf9d0f12961f5
author_association: NONE
user: login: QodoAI-Agent
created_at: 2025-05-02T07:41:33Z
body: **[Persistent review](https://github.com/truffle-ai/saiki/pull/85#issuecomment-2844065841)** updated to latest commit https://github.com/truffle-ai/saiki/commit/97f6c9189689d483a0b031a7d3dad47147419892
author_association: NONE