Repository: meta-llama/llama-stack
Pull request #1408: feat: add support for logging config in the run.yaml
Author: login: cdoern
Labels: name: CLA Signed
description: This label is managed by the Meta Open Source bot.
Created: 2025-03-05T03:31:19Z | Merged: 2025-03-14T19:36:25Z

# What does this PR do?

a user should be able to store a static logging configuration outside of their environment. This would make sense to store in the run yaml given that we store other things like server configuration in there.

The environment variable settings override the config settings if both are available.

The format in the config looks like this:

```
logging_config:
  category_levels:
    VALID_CATEGORY: VALID_STRING_LOG_LEVEL
```

any specified category out of the following:

`core | server | router | inference | agents | safety | eval | tools | client`

combined with any of the following log levels:

`debug | info | warning | error | critical`

can be placed in the category_levels list in order to achieve the desired log level

## Test Plan

Test locally with a run config like the following:

```
version: '2'
image_name: ollama
logging_config:
  category_levels:
      server: debug
apis:
...
```



Diff:
diff --git a/llama_stack/distribution/datatypes.py b/llama_stack/distribution/datatypes.py
index f62996081b..7e1d8c016c 100644
--- a/llama_stack/distribution/datatypes.py
+++ b/llama_stack/distribution/datatypes.py
@@ -117,6 +117,14 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LoggingConfig(BaseModel):
+    category_levels: Dict[str, str] = Field(
+        default_factory=Dict,
+        description="""
+ Dictionary of different logging configurations for different portions (ex: core, server) of llama stack""",
+    )
+
+
 class ServerConfig(BaseModel):
     port: int = Field(
         default=8321,
@@ -176,6 +184,8 @@ class StackRunConfig(BaseModel):
     benchmarks: List[BenchmarkInput] = Field(default_factory=list)
     tool_groups: List[ToolGroupInput] = Field(default_factory=list)
 
+    logging: Optional[LoggingConfig] = Field(default=None, description="Configuration for Llama Stack Logging")
+
     server: ServerConfig = Field(
         default_factory=ServerConfig,
         description="Configuration for the HTTP(S) server",
diff --git a/llama_stack/distribution/server/server.py b/llama_stack/distribution/server/server.py
index 8f9500ae9b..b37b3a007d 100644
--- a/llama_stack/distribution/server/server.py
+++ b/llama_stack/distribution/server/server.py
@@ -25,7 +25,7 @@
 from pydantic import BaseModel, ValidationError
 from typing_extensions import Annotated
 
-from llama_stack.distribution.datatypes import StackRunConfig
+from llama_stack.distribution.datatypes import LoggingConfig, StackRunConfig
 from llama_stack.distribution.distribution import builtin_automatically_routed_apis
 from llama_stack.distribution.request_headers import (
     PROVIDER_DATA_VAR,
@@ -306,34 +306,42 @@ def main():
 
     args = parser.parse_args()
 
-    if args.env:
-        for env_pair in args.env:
-            try:
-                key, value = validate_env_pair(env_pair)
-                logger.info(f"Setting CLI environment variable {key} => {value}")
-                os.environ[key] = value
-            except ValueError as e:
-                logger.error(f"Error: {str(e)}")
-                sys.exit(1)
-
+    log_line = ""
     if args.yaml_config:
         # if the user provided a config file, use it, even if template was specified
         config_file = Path(args.yaml_config)
         if not config_file.exists():
             raise ValueError(f"Config file {config_file} does not exist")
-        logger.info(f"Using config file: {config_file}")
+        log_line = f"Using config file: {config_file}"
     elif args.template:
         config_file = Path(REPO_ROOT) / "llama_stack" / "templates" / args.template / "run.yaml"
         if not config_file.exists():
             raise ValueError(f"Template {args.template} does not exist")
-        logger.info(f"Using template {args.template} config file: {config_file}")
+        log_line = f"Using template {args.template} config file: {config_file}"
     else:
         raise ValueError("Either --yaml-config or --template must be provided")
 
+    logger_config = None
     with open(config_file, "r") as fp:
-        config = replace_env_vars(yaml.safe_load(fp))
+        config_contents = yaml.safe_load(fp)
+        if isinstance(config_contents, dict) and (cfg := config_contents.get("logging_config")):
+            logger_config = LoggingConfig(**cfg)
+        logger = get_logger(name=__name__, category="server", config=logger_config)
+        if args.env:
+            for env_pair in args.env:
+                try:
+                    key, value = validate_env_pair(env_pair)
+                    logger.info(f"Setting CLI environment variable {key} => {value}")
+                    os.environ[key] = value
+                except ValueError as e:
+                    logger.error(f"Error: {str(e)}")
+                    sys.exit(1)
+        config = replace_env_vars(config_contents)
         config = StackRunConfig(**config)
 
+    # now that the logger is initialized, print the line about which type of config we are using.
+    logger.info(log_line)
+
     logger.info("Run configuration:")
     safe_config = redact_sensitive_fields(config.model_dump())
     logger.info(yaml.dump(safe_config, indent=2))
diff --git a/llama_stack/log.py b/llama_stack/log.py
index 572dea234f..0ba95d547b 100644
--- a/llama_stack/log.py
+++ b/llama_stack/log.py
@@ -7,13 +7,15 @@
 import logging
 import os
 from logging.config import dictConfig
-from typing import Dict
+from typing import Dict, Optional
 
 from rich.console import Console
 from rich.errors import MarkupError
 from rich.logging import RichHandler
 from termcolor import cprint
 
+from .distribution.datatypes import LoggingConfig
+
 # Default log level
 DEFAULT_LOG_LEVEL = logging.INFO
 
@@ -34,6 +36,56 @@
 _category_levels: Dict[str, int] = {category: DEFAULT_LOG_LEVEL for category in CATEGORIES}
 
 
+def config_to_category_levels(category: str, level: str):
+    """
+    Helper function to be called either by environment parsing or yaml parsing to go from a list of categories and levels to a dictionary ready to be
+    used by the logger dictConfig.
+
+    Parameters:
+        category (str): logging category to apply the level to
+        level (str): logging level to be used in the category
+
+    Returns:
+        Dict[str, int]: A dictionary mapping categories to their log levels.
+    """
+
+    category_levels: Dict[str, int] = {}
+    level_value = logging._nameToLevel.get(str(level).upper())
+    if level_value is None:
+        logging.warning(f"Unknown log level '{level}' for category '{category}'. Falling back to default 'INFO'.")
+        return category_levels
+
+    if category == "all":
+        # Apply the log level to all categories and the root logger
+        for cat in CATEGORIES:
+            category_levels[cat] = level_value
+        # Set the root logger's level to the specified level
+        category_levels["root"] = level_value
+    elif category in CATEGORIES:
+        category_levels[category] = level_value
+        logging.info(f"Setting '{category}' category to level '{level}'.")
+    else:
+        logging.warning(f"Unknown logging category: {category}. No changes made.")
+    return category_levels
+
+
+def parse_yaml_config(yaml_config: LoggingConfig) -> Dict[str, int]:
+    """
+    Helper function to parse a yaml logging configuration found in the run.yaml
+
+    Parameters:
+        yaml_config (Logging): the logger config object found in the run.yaml
+
+    Returns:
+        Dict[str, int]: A dictionary mapping categories to their log levels.
+    """
+    category_levels = {}
+    for category, level in yaml_config.category_levels.items():
+        category_levels.update(config_to_category_levels(category=category, level=level))
+
+    return category_levels
+
+
 def parse_environment_config(env_config: str) -> Dict[str, int]:
     """
     Parse the LLAMA_STACK_LOGGING environment variable and return a dictionary of category log levels.
@@ -53,25 +105,7 @@ def parse_environment_config(env_config: str) -> Dict[str, int]:
             category, level = pair.split("=", 1)
             category = category.strip().lower()
             level = level.strip().upper()  # Convert to uppercase for logging._nameToLevel
-
-            level_value = logging._nameToLevel.get(level)
-            if level_value is None:
-                logging.warning(
-                    f"Unknown log level '{level}' for category '{category}'. Falling back to default 'INFO'."
-                )
-                continue
-
-            if category == "all":
-                # Apply the log level to all categories and the root logger
-                for cat in CATEGORIES:
-                    category_levels[cat] = level_value
-                # Set the root logger's level to the specified level
-                category_levels["root"] = level_value
-            elif category in CATEGORIES:
-                category_levels[category] = level_value
-                logging.info(f"Setting '{category}' category to level '{level}'.")
-            else:
-                logging.warning(f"Unknown logging category: {category}. No changes made.")
+            category_levels.update(config_to_category_levels(category=category, level=level))
 
         except ValueError:
             logging.warning(f"Invalid logging configuration: '{pair}'. Expected format: 'category=level'.")
@@ -176,7 +210,9 @@ def filter(self, record):
             logger.setLevel(root_level)
 
 
-def get_logger(name: str, category: str = "uncategorized") -> logging.LoggerAdapter:
+def get_logger(
+    name: str, category: str = "uncategorized", config: Optional[LoggingConfig] | None = None
+) -> logging.LoggerAdapter:
     """
     Returns a logger with the specified name and category.
     If no category is provided, defaults to 'uncategorized'.
@@ -184,10 +220,14 @@ def get_logger(name: str, category: str = "uncategorized") -> logging.LoggerAdap
     Parameters:
         name (str): The name of the logger (e.g., module or filename).
         category (str): The category of the logger (default 'uncategorized').
+        config (Logging): optional yaml config to override the existing logger configuration
 
     Returns:
         logging.LoggerAdapter: Configured logger with category support.
     """
+    if config:
+        _category_levels.update(parse_yaml_config(config))
+
     logger = logging.getLogger(name)
     logger.setLevel(_category_levels.get(category, DEFAULT_LOG_LEVEL))
     return logging.LoggerAdapter(logger, {"category": category})


Reviews:
user: login: nathan-weinberg
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T14:11:45Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-12T16:43:47Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-12T16:43:54Z
user: login: ashwinb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T01:22:24Z
user: login: ashwinb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T01:23:11Z
user: login: ashwinb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T01:58:37Z
user: login: ashwinb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T01:59:48Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T11:34:12Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T11:34:18Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T11:35:21Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T12:38:37Z
user: login: ashwinb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T18:46:37Z
user: login: cdoern
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T22:24:22Z
user: login: ashwinb
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T19:36:17Z

Review comments:
diff_hunk: @@ -117,6 +117,22 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LogCategoryConfig(BaseModel):
+    category: str = Field(
+        default="server",
+        description="logging category to apply this configuration to (core | server | router | inference | agents | safety | eval | tools | client)",
+    )
+    level: str = Field(default="info", description="Log level (debug | info | warning | error | critical)")
+
+
+class LoggerConfig(BaseModel):
+    category_levels: List[LogCategoryConfig] = Field(
+        default_factory=list,
+        description="""
+The list different logging configurations for different portions of llama stack""",
path: llama_stack/distribution/datatypes.py
user: login: nathan-weinberg
body: Don't quite understand this text
created_at: 2025-03-12T14:05:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -302,34 +302,42 @@ def main():
 
     args = parser.parse_args()
 
-    if args.env:
-        for env_pair in args.env:
-            try:
-                key, value = validate_env_pair(env_pair)
-                logger.info(f"Setting CLI environment variable {key} => {value}")
-                os.environ[key] = value
-            except ValueError as e:
-                logger.error(f"Error: {str(e)}")
-                sys.exit(1)
-
+    log_string = ""
path: llama_stack/distribution/server/server.py
user: login: nathan-weinberg
body: suggest `log_source_info` instead of `log_string`
created_at: 2025-03-12T14:10:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -117,6 +117,22 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LogCategoryConfig(BaseModel):
+    category: str = Field(
+        default="server",
+        description="logging category to apply this configuration to (core | server | router | inference | agents | safety | eval | tools | client)",
path: llama_stack/distribution/datatypes.py
user: login: nathan-weinberg
body: It seems from the below code that `all` is also a valid option?
created_at: 2025-03-12T14:11:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -117,6 +117,22 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LogCategoryConfig(BaseModel):
+    category: str = Field(
+        default="server",
+        description="logging category to apply this configuration to (core | server | router | inference | agents | safety | eval | tools | client)",
+    )
+    level: str = Field(default="info", description="Log level (debug | info | warning | error | critical)")
+
+
+class LoggerConfig(BaseModel):
+    category_levels: List[LogCategoryConfig] = Field(
+        default_factory=list,
+        description="""
+The list different logging configurations for different portions of llama stack""",
path: llama_stack/distribution/datatypes.py
user: login: cdoern
body: adjusted a bit
created_at: 2025-03-12T16:43:47Z
author_association: COLLABORATOR
diff_hunk: @@ -117,6 +117,22 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LogCategoryConfig(BaseModel):
+    category: str = Field(
+        default="server",
+        description="logging category to apply this configuration to (core | server | router | inference | agents | safety | eval | tools | client)",
path: llama_stack/distribution/datatypes.py
user: login: cdoern
body: yup, added that in
created_at: 2025-03-12T16:43:53Z
author_association: COLLABORATOR
diff_hunk: @@ -117,6 +117,22 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LogCategoryConfig(BaseModel):
+    category: str = Field(
+        default="server",
+        description="Logging category to apply this configuration to (all | core | server | router | inference | agents | safety | eval | tools | client)",
+    )
+    level: str = Field(default="info", description="Log level (debug | info | warning | error | critical)")
+
+
+class LoggerConfig(BaseModel):
+    category_levels: List[LogCategoryConfig] = Field(
path: llama_stack/distribution/datatypes.py
user: login: ashwinb
body: could this be a `dict`? `category => level`?
created_at: 2025-03-13T01:22:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -176,6 +192,8 @@ class StackRunConfig(BaseModel):
     benchmarks: List[BenchmarkInput] = Field(default_factory=list)
     tool_groups: List[ToolGroupInput] = Field(default_factory=list)
 
+    logger_config: Optional[LoggerConfig] = Field(default=None, description="Configuration of the Llama Stack Logger")
path: llama_stack/distribution/datatypes.py
user: login: ashwinb
body: could the key be called `logging` (since this is nested under a "ServerConfig" structure already)? 
created_at: 2025-03-13T01:23:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -302,34 +302,42 @@ def main():
 
     args = parser.parse_args()
 
-    if args.env:
-        for env_pair in args.env:
-            try:
-                key, value = validate_env_pair(env_pair)
-                logger.info(f"Setting CLI environment variable {key} => {value}")
-                os.environ[key] = value
-            except ValueError as e:
-                logger.error(f"Error: {str(e)}")
-                sys.exit(1)
-
+    log_source_info = ""
     if args.yaml_config:
         # if the user provided a config file, use it, even if template was specified
         config_file = Path(args.yaml_config)
         if not config_file.exists():
             raise ValueError(f"Config file {config_file} does not exist")
-        logger.info(f"Using config file: {config_file}")
+        log_source_info = f"Using config file: {config_file}"
path: llama_stack/distribution/server/server.py
user: login: ashwinb
body: nit: maybe just rename `log_line` ?
created_at: 2025-03-13T01:58:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -302,34 +302,42 @@ def main():
 
     args = parser.parse_args()
 
-    if args.env:
-        for env_pair in args.env:
-            try:
-                key, value = validate_env_pair(env_pair)
-                logger.info(f"Setting CLI environment variable {key} => {value}")
-                os.environ[key] = value
-            except ValueError as e:
-                logger.error(f"Error: {str(e)}")
-                sys.exit(1)
-
+    log_source_info = ""
     if args.yaml_config:
         # if the user provided a config file, use it, even if template was specified
         config_file = Path(args.yaml_config)
         if not config_file.exists():
             raise ValueError(f"Config file {config_file} does not exist")
-        logger.info(f"Using config file: {config_file}")
+        log_source_info = f"Using config file: {config_file}"
     elif args.template:
         config_file = Path(REPO_ROOT) / "llama_stack" / "templates" / args.template / "run.yaml"
         if not config_file.exists():
             raise ValueError(f"Template {args.template} does not exist")
-        logger.info(f"Using template {args.template} config file: {config_file}")
+        log_source_info = f"Using template {args.template} config file: {config_file}"
     else:
         raise ValueError("Either --yaml-config or --template must be provided")
 
+    logger_config = None
     with open(config_file, "r") as fp:
-        config = replace_env_vars(yaml.safe_load(fp))
+        config_contents = yaml.safe_load(fp)
+        if config_contents["logger_config"]:
path: llama_stack/distribution/server/server.py
user: login: ashwinb
body: the key may not exist. suggest:

```python
if isinstance(config_contents, dict) and cfg := config_contents.get("logger_config"):
   logger_config = LoggerConfig(**cfg)
```
created_at: 2025-03-13T01:59:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -176,6 +192,8 @@ class StackRunConfig(BaseModel):
     benchmarks: List[BenchmarkInput] = Field(default_factory=list)
     tool_groups: List[ToolGroupInput] = Field(default_factory=list)
 
+    logger_config: Optional[LoggerConfig] = Field(default=None, description="Configuration of the Llama Stack Logger")
path: llama_stack/distribution/datatypes.py
user: login: cdoern
body: sure, renamed to logging
created_at: 2025-03-13T11:34:12Z
author_association: COLLABORATOR
diff_hunk: @@ -117,6 +117,22 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class LogCategoryConfig(BaseModel):
+    category: str = Field(
+        default="server",
+        description="Logging category to apply this configuration to (all | core | server | router | inference | agents | safety | eval | tools | client)",
+    )
+    level: str = Field(default="info", description="Log level (debug | info | warning | error | critical)")
+
+
+class LoggerConfig(BaseModel):
+    category_levels: List[LogCategoryConfig] = Field(
path: llama_stack/distribution/datatypes.py
user: login: cdoern
body: yup, changed to that
created_at: 2025-03-13T11:34:18Z
author_association: COLLABORATOR
diff_hunk: @@ -302,34 +302,42 @@ def main():
 
     args = parser.parse_args()
 
-    if args.env:
-        for env_pair in args.env:
-            try:
-                key, value = validate_env_pair(env_pair)
-                logger.info(f"Setting CLI environment variable {key} => {value}")
-                os.environ[key] = value
-            except ValueError as e:
-                logger.error(f"Error: {str(e)}")
-                sys.exit(1)
-
+    log_source_info = ""
     if args.yaml_config:
         # if the user provided a config file, use it, even if template was specified
         config_file = Path(args.yaml_config)
         if not config_file.exists():
             raise ValueError(f"Config file {config_file} does not exist")
-        logger.info(f"Using config file: {config_file}")
+        log_source_info = f"Using config file: {config_file}"
path: llama_stack/distribution/server/server.py
user: login: cdoern
body: yeah that makes more sense
created_at: 2025-03-13T11:35:21Z
author_association: COLLABORATOR
diff_hunk: @@ -302,34 +302,42 @@ def main():
 
     args = parser.parse_args()
 
-    if args.env:
-        for env_pair in args.env:
-            try:
-                key, value = validate_env_pair(env_pair)
-                logger.info(f"Setting CLI environment variable {key} => {value}")
-                os.environ[key] = value
-            except ValueError as e:
-                logger.error(f"Error: {str(e)}")
-                sys.exit(1)
-
+    log_source_info = ""
     if args.yaml_config:
         # if the user provided a config file, use it, even if template was specified
         config_file = Path(args.yaml_config)
         if not config_file.exists():
             raise ValueError(f"Config file {config_file} does not exist")
-        logger.info(f"Using config file: {config_file}")
+        log_source_info = f"Using config file: {config_file}"
     elif args.template:
         config_file = Path(REPO_ROOT) / "llama_stack" / "templates" / args.template / "run.yaml"
         if not config_file.exists():
             raise ValueError(f"Template {args.template} does not exist")
-        logger.info(f"Using template {args.template} config file: {config_file}")
+        log_source_info = f"Using template {args.template} config file: {config_file}"
     else:
         raise ValueError("Either --yaml-config or --template must be provided")
 
+    logger_config = None
     with open(config_file, "r") as fp:
-        config = replace_env_vars(yaml.safe_load(fp))
+        config_contents = yaml.safe_load(fp)
+        if config_contents["logger_config"]:
path: llama_stack/distribution/server/server.py
user: login: cdoern
body: took this, added some parenthesis to get it to parse properly, thanks!
created_at: 2025-03-13T12:38:37Z
author_association: COLLABORATOR
diff_hunk: @@ -117,6 +117,14 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class Logging(BaseModel):
path: llama_stack/distribution/datatypes.py
user: login: ashwinb
body: oh I meant just change the `key` not the classname. so `logging: Optional[LoggingConfig]`, sorry for being unclear and somewhat nitpicky.
created_at: 2025-03-13T18:46:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -117,6 +117,14 @@ class Provider(BaseModel):
     config: Dict[str, Any]
 
 
+class Logging(BaseModel):
path: llama_stack/distribution/datatypes.py
user: login: cdoern
body: no worries, just changed it back 
created_at: 2025-03-13T22:24:22Z
author_association: COLLABORATOR

PR comments:
user: login: cdoern
created_at: 2025-03-05T03:31:58Z
body: @leseb happy to rebase this on top of your logging work once that gets in, but I figured I'd put this up with `logcat` as it is right now!
author_association: COLLABORATOR
user: login: leseb
created_at: 2025-03-05T16:03:35Z
body: > @leseb happy to rebase this on top of your logging work once that gets in, but I figured I'd put this up with `logcat` as it is right now!

Thanks for the heads-up; yeah hopefully https://github.com/meta-llama/llama-stack/pull/1362 can be merged first :)
author_association: COLLABORATOR
user: login: nathan-weinberg
created_at: 2025-03-11T14:22:26Z
body: @cdoern the #1362 PR by @leseb has been merged - if you can rebase on top of current `main` I'm happy to give this a review!
author_association: CONTRIBUTOR
user: login: cdoern
created_at: 2025-03-11T18:49:36Z
body: @nathan-weinberg should be good to review now. 
author_association: COLLABORATOR