Repository: DAGWorks-Inc/hamilton
Pull request #1294: Add Context-Aware Synchronous/Asynchronous Logging Adapters
Author: login: cswartzvi
Created: 2025-03-17T04:19:47Z | Merged: 2025-03-29T18:15:31Z

This PR is a follow-on to #1287 utilizing the new task submission and resolution hooks to create context-aware logging adapters.

Disclaimers:
- I fully realize this may be too much for base Hamilton. I have been using a version of this at work for a few months and it performs well. Please do not feel any sort of bad if this is rejected - I can easily continue to use it as a separate extension
- The details for context-aware logging, and structure of the logs, were larger inspired by `prefect`
- This branch is based on #1287 and will need to be rebased if/when accepted

## Changes

This PR adds a pair of adapters (in `hamiton.plugins.h_logging`) named `LoggingAdapter` and `AsyncLoggingAdapter` that can be used to log the following state in a the execution of a graph (supporting both V1/V2 drivers where applicable):
  - Graph start (`GraphExecutionHook`)
  - Task grouping (`TaskGroupingHook`)
  - Task submission (`TaskSubmissionHook`)
  - Task pre-execution (`TaskExecutionHook`))
  - Node pre-execution (`NodeExecutionHook`)
  - Node post-execution (`NodeExecutionHook`)
  - Task post-execution (`TaskExecutionHook`)
  - Task resolution  (`TaskResolutionHook`)
  - Graph completion  (`GraphExecutionHook`)

These adapters keep track of their current execution context by using an internal `ContextVar`. This allows the log to have a context dependent prefix, by way of an internal custom log adapter, such as 
```
Graph run 'c7236c13-94ca-4e5e-85a6-2f32af054736' - Starting graph execution 
```
or 

```
`Task 'expand-stargazer_url.0.block-stargazer_url' - Task completed [OK]
```

Additionally, a function called `get_logger` was added that returns the custom log adapter which the user can use to create context-aware logs from within a node. For example the following log inside node `a`...

```python
from hamilton.plugins.h_logging import get_logger

def a() -> str:
    logger = get_logger("name_or_logger_or_none")
    logger.warning("Encountered a warning")
```

will generate the following:

```
Node 'a' - Encountered a warning
```

See the Notes section for some examples


## How I tested this

I added tests for both `LoggingAdapter` and `AsyncLoggingAdapter`. The tests for `LoggingAdapter` examine the V1 driver with and without the `FutureAdapter` and the V2 driver with the synchronous, multi-threading, multi-process, ray, and dask task-based executors. The tests may be a little brittle because some are dependent on undocumented task id naming conventions. Open to suggestions 😄.

## Examples

Here I would like to present some examples. I used a logging configuration based on rich - but any will suffice.

### Synchronous - non-branching

Standard node-based Hamilton graphs using the `LoggingAdapter`...

```python
def a() -> str:
    return "a"

def b(a: str) -> str:
    return a + " b"

def c(b: str) -> str:
    return b + " c"
```
will produce the following ...

```
INFO     Graph run '24fd50b2-760b-4cf2-b1e1-61476c18e9b3' - Starting graph execution
DEBUG    Node 'a' - Starting execution without dependencies
INFO     Node 'a' - Finished execution [OK]
DEBUG    Node 'b' - Starting execution with dependencies 'a'
INFO     Node 'b' - Finished execution [OK]
DEBUG    Node 'c' - Starting execution with dependencies 'b'
INFO     Node 'c' - Finished execution [OK]
INFO     Graph run '24fd50b2-760b-4cf2-b1e1-61476c18e9b3' - Finished graph execution [OK]
```

### Synchronous - branching

Branching graphs with multiple possible paths using the `LoggingAdapter` (with or without the `FutureAdapter`)...

```python
    def a() -> str:
        return "a"

    def b() -> str:
        return "b"

    def c() -> str:
        return "c"

    def d(a: str, b: str) -> str:
        return a + " " + b + " d"

    def e(c: str) -> str:
        return c + " e"

    def f(d: str, e: str) -> str:
        return d + " " + e + " f"
```

will produce the following (with a potentially different order)...

```
INFO     Graph run '31d4bc1d-8020-4e24-91a5-732a03497f1a' - Starting graph execution
DEBUG    Node 'c' - Submitting async node without dependencies
DEBUG    Node 'a' - Submitting async node without dependencies
DEBUG    Node 'b' - Submitting async node without dependencies
DEBUG    Node 'd' - Submitting async node with dependencies 'a', 'b'
DEBUG    Node 'e' - Submitting async node with dependencies 'c'
DEBUG    Node 'f' - Submitting async node with dependencies 'd', 'e'
INFO     Node 'c' - Finished execution [OK]
INFO     Node 'a' - Finished execution [OK]
INFO     Node 'b' - Finished execution [OK]
INFO     Node 'e' - Finished execution [OK]
INFO     Node 'd' - Finished execution [OK]
INFO     Node 'f' - Finished execution [OK]
INFO     Graph run '31d4bc1d-8020-4e24-91a5-732a03497f1a' - Finished graph execution [OK]
```

### Asynchronous - Branching

Async branching graphs are also supported using the `AsyncLoggingAdapter` but **must** be used with the async driver. There is a little weakness in this adapter dealing with the current state of hooks in the async adapter - see the code for more details.

```python
async def a() -> str:
    return "a"

async def b() -> str:
    return "b"

async def c() -> str:
    return "c"

async def d(a: str, b: str) -> str:
    return a + " " + b + " d"

async def e(c: str) -> str:
    return c + " e"

async def f(d: str, e: str) -> str:
    return d + " " + e + " f"
``` 

These will produce the following (again with a potentially different order)...

```INFO     Graph run '38deee89-4a91-4253-b913-ce3c1e60b791' - Starting graph execution
DEBUG    Node 'a' - Submitting async node without dependencies
DEBUG    Node 'b' - Submitting async node without dependencies
DEBUG    Node 'd' - Submitting async node with dependencies 'a', 'b'
DEBUG    Node 'c' - Submitting async node without dependencies
DEBUG    Node 'e' - Submitting async node with dependencies 'c'
DEBUG    Node 'f' - Submitting async node with dependencies 'd', 'e'
INFO     Node 'a' - Finished execution [OK]
INFO     Node 'b' - Finished execution [OK]
INFO     Node 'c' - Finished execution [OK]
INFO     Node 'd' - Finished execution [OK]
INFO     Node 'e' - Finished execution [OK]
INFO     Node 'f' - Finished execution [OK]
INFO     Graph run '38deee89-4a91-4253-b913-ce3c1e60b791' - Finished graph execution [OK]
```

### Task based

Task-based executors (synchronous, threading, processing, ray, dask ,...) are also supported with the `LoggingAdapter`. Note however that user context logs from inside a node may (threading, ray) or may not (multiprocessing, dask) be supported. Some executors provide log pass throughs (ray) the produce slightly different results. 

```python
def b(a: int) -> int:
    return a

def c(b: int) -> Parallelizable[int]:
    for i in range(b):
        yield i

def d(c: int) -> int:
    return 2 * c

def e(d: Collect[int]) -> int:
    return sum(d)

def f(e: int) -> int:
    return e
```

These will produce the following (with a potentially different order) when `a = 2`...

```
INFO     Graph run 'dc4ec112-ba50-44ae-8b64-11b0ad099f74' - Starting graph execution
INFO     Graph run 'dc4ec112-ba50-44ae-8b64-11b0ad099f74' - Using inputs 'a'
INFO     Graph run 'dc4ec112-ba50-44ae-8b64-11b0ad099f74' - Dynamic DAG detected; task-based logging is enabled
DEBUG    Task 'a' - Initializing new task and submitting to executor
DEBUG    Task 'a' - Task completed [OK]
DEBUG    Task 'b' - Initializing new task and submitting to executor
DEBUG    Task 'b' - Starting execution
DEBUG    Task 'b' - Starting execution with dependencies 'a'
DEBUG    Task 'b' - Node 'b' - Finished execution [OK]
DEBUG    Task 'b' - Finished execution [Ok]
INFO     Task 'b' - Task completed [OK]
DEBUG    Task 'expand-c' - Initializing new task and submitting to executor
DEBUG    Task 'expand-c' - Starting execution of nodes 'c'
DEBUG    Task 'expand-c' - Starting execution with dependencies 'b'
DEBUG    Task 'expand-c' - Node 'c' - Finished execution [OK]
DEBUG    Task 'expand-c' - Finished execution [Ok]
INFO     Task 'expand-c' - Task completed [OK]
DEBUG    Task 'expand-c.0.block-c' - Spawning task and submitting to executor
DEBUG    Task 'expand-c.0.block-c' - Starting execution of nodes 'd'
DEBUG    Task 'expand-c.0.block-c' - Starting execution with dependencies 'c'
DEBUG    Task 'expand-c.0.block-c' - Node 'd' - Finished execution [OK]
DEBUG    Task 'expand-c.0.block-c' - Finished execution [Ok]
INFO     Task 'expand-c.0.block-c' - Task completed [OK]
DEBUG    Task 'expand-c.1.block-c' - Spawning task and submitting to executor
DEBUG    Task 'expand-c.1.block-c' - Starting execution of nodes 'd'
DEBUG    Task 'expand-c.1.block-c' - Starting execution with dependencies 'c'
DEBUG    Task 'expand-c.1.block-c' - Node 'd' - Finished execution [OK]
DEBUG    Task 'expand-c.1.block-c' - Finished execution [Ok]
INFO     Task 'expand-c.1.block-c' - Task completed [OK]
DEBUG    Task 'collect-c' - Initializing new task and submitting to executor
DEBUG    Task 'collect-c' - Starting execution of nodes 'e'
DEBUG    Task 'collect-c' - Starting execution with dependencies 'd'
DEBUG    Task 'collect-c' - Node 'e' - Finished execution [OK]
DEBUG    Task 'collect-c' - Finished execution [Ok]
INFO     Task 'collect-c' - Task completed [OK]
DEBUG    Task 'f' - Initializing new task and submitting to executor
DEBUG    Task 'f' - Starting execution
DEBUG    Task 'f' - Starting execution with dependencies 'e'
DEBUG    Task 'f' - Node 'f' - Finished execution [OK]
DEBUG    Task 'f' - Finished execution [Ok]
INFO     Task 'f' - Task completed [OK]
INFO     Graph run 'dc4ec112-ba50-44ae-8b64-11b0ad099f74' - Finished graph execution [OK]
```

Sorry that was alot! 😅 If you like what you see, please let me know! Note that I have **not** added docs for this yet - happily planning to do so after an initial review!

## Checklist

- [x] PR has an informative and human-readable title (this will be pulled into the release notes)
- [x] Changes are limited to a single goal (no scope creep)
- [x] Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
- [x] Any _change_ in functionality is tested
- [ ] New functions are documented (with a description, list of inputs, and expected output)
- [x] Placeholder code is flagged / future TODOs are captured in comments
- [ ] Project documentation has been updated if adding/changing functionality.



Diff:
diff --git a/hamilton/lifecycle/api.py b/hamilton/lifecycle/api.py
index 98db5a9c1..63e176264 100644
--- a/hamilton/lifecycle/api.py
+++ b/hamilton/lifecycle/api.py
@@ -764,7 +764,7 @@ def post_task_expand(self, *, run_id: str, task_id: str, parameters: Dict[str, A
 
     @abc.abstractmethod
     def run_after_task_grouping(self, *, run_id: str, task_ids: List[str], **future_kwargs):
-        """Runs after task grouping. This allows you to capture information about the which tasks were
+        """Runs after task grouping. This allows you to capture information about which tasks were
         created for a given run.
 
         :param run_id: ID of the run, unique in scope of the driver.
diff --git a/hamilton/plugins/h_logging.py b/hamilton/plugins/h_logging.py
new file mode 100644
index 000000000..542b6c425
--- /dev/null
+++ b/hamilton/plugins/h_logging.py
@@ -0,0 +1,444 @@
+"""Synchronous/asynchronous adapter and functions for context-aware logging with Hamilton."""
+
+import logging
+import sys
+from contextvars import ContextVar
+from dataclasses import dataclass
+from typing import (
+    TYPE_CHECKING,
+    Any,
+    Dict,
+    List,
+    Mapping,
+    MutableMapping,
+    Optional,
+    Set,
+    Tuple,
+    Union,
+)
+
+from hamilton.graph_types import HamiltonNode
+from hamilton.lifecycle.api import (
+    GraphExecutionHook,
+    NodeExecutionHook,
+    TaskExecutionHook,
+    TaskGroupingHook,
+    TaskReturnHook,
+    TaskSubmissionHook,
+)
+from hamilton.lifecycle.base import BasePostNodeExecuteAsync, BasePreNodeExecute
+from hamilton.node import Node
+
+try:
+    from typing import override
+except ImportError:
+    override = lambda x: x  # noqa E731
+
+
+if sys.version_info >= (3, 12):
+    LoggerAdapter = logging.LoggerAdapter[logging.Logger]
+else:
+    if TYPE_CHECKING:
+        LoggerAdapter = logging.LoggerAdapter[logging.Logger]
+    else:
+        LoggerAdapter = logging.LoggerAdapter
+
+
+@dataclass(frozen=True)
+class _LoggingContext:
+    """Represents the current logging context."""
+
+    graph: Optional[str] = None
+    node: Optional[str] = None
+    task: Optional[str] = None
+
+
+# Context variables for context-aware logging
+_local_context = ContextVar("context", default=_LoggingContext())
+
+
+def get_logger(name: Optional[str] = None) -> "ContextLogger":
+    """Returns a context-aware logger for the specified name (created if necessary).
+
+    :param name: Name of the logger, defaults to root logger if not provided.
+    """
+    logger = logging.getLogger(name)
+    return ContextLogger(logger, extra={})
+
+
+class ContextLogger(LoggerAdapter):
+    """Custom logger adapter for Hamilton that adds context to log messages.
+
+    This logger adds context-aware prefix to log messages based on the current execution. The
+    logger is intended to be used with hamilton the `LoggingAdapter` lifecycle adapter. The context
+    is both thread-safe and async-safe. Context includes the following details:
+    - Graph run
+    - Task ID
+    - Node ID
+
+    The adapter also supports the following extra fields:
+    - `override_context`: Overrides the current context with the specified context.
+    - `skip_context`: Skips the context for the current log message.
+    - `node_context`: Includes additional node information for task-based log messages.
+    """
+
+    @override
+    def process(
+        self, msg: str, kwargs: MutableMapping[str, Any]
+    ) -> Tuple[str, MutableMapping[str, Any]]:
+        # Ensure that the extra fields are passed through correctly
+        kwargs["extra"] = {**(self.extra or {}), **(kwargs.get("extra") or {})}
+
+        # Add the current prefix to the log message
+        prefix = self._get_current_context(kwargs["extra"])
+        msg = f"{prefix}{msg}"
+
+        return (msg, kwargs)
+
+    def _get_current_context(self, extra: Mapping[str, Any]) -> str:
+        """Returns the current context."""
+
+        # Extra option to override context
+        context = extra.get("override_context", None)
+        if not isinstance(context, _LoggingContext):
+            context = _local_context.get()
+
+        # Extra option to skip context
+        if "skip_context" in extra:
+            return ""
+
+        if context.task:
+            # Extra option to include node information on task-based log messages
+            if context.node and "node_context" in extra:
+                return f"Task '{context.task}' - Node '{context.node}' - "
+            return f"Task '{context.task}' - "
+
+        if context.node:
+            return f"Node '{context.node}' - "
+
+        if context.graph:
+            return f"Graph run '{context.graph}' - "
+
+        return ""
+
+
+class LoggingAdapter(
+    GraphExecutionHook,
+    NodeExecutionHook,
+    TaskGroupingHook,
+    TaskSubmissionHook,
+    TaskExecutionHook,
+    TaskReturnHook,
+):
+    """Hamilton lifecycle adapter that logs runtime execution events.
+
+    This adapter logs the following hamilton events:
+    - Graph start (`GraphExecutionHook`)
+    - Task grouping (`TaskGroupingHook`)
+    - Task submission (`TaskSubmissionHook`)
+    - Task pre-execution (`TaskExecutionHook`))
+    - Node pre-execution (`NodeExecutionHook`)
+    - Node post-execution (`NodeExecutionHook`)
+    - Task post-execution (`TaskExecutionHook`))
+    - Task resolution  (`TaskReturnHook`)
+    - Graph completion  (`GraphExecutionHook`)
+
+    This adapter can be run with both node-based and task-based execution (using the V2 executor).
+    When run with a node-based executor, the adapter logs the execution of each *node* as `INFO`.
+    When run with a task-based executor, the adapter logs the execution of each *task* as `INFO`
+    and the execution of each *node* as `DEBUG`.
+    """
+
+    def __init__(self, logger: Union[str, logging.Logger, None] = None) -> None:
+        # Precompute or overridden nodes
+        self._inputs_nodes: Set[str] = set()
+        self._override_nodes: Set[str] = set()
+
+        if logger is None:
+            self.logger = logging.getLogger(__name__)
+        elif isinstance(logger, logging.Logger):
+            self.logger = logger
+        else:
+            self.logger = logging.getLogger(logger)
+
+        if not isinstance(self.logger, ContextLogger):
+            self.logger = ContextLogger(self.logger, extra={})
+
+        self._exception_logged = False  # For tracking remote exceptions
+
+    @override
+    def run_before_graph_execution(
+        self,
+        *,
+        inputs: Optional[Dict[str, Any]],
+        overrides: Optional[Dict[str, Any]],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id))
+
+        self._inputs_nodes.update(inputs or [])
+        self._override_nodes.update(overrides or [])
+
+        message = "Starting graph execution"
+        self.logger.info(message)
+
+        if inputs:
+            names = ", ".join(f"'{key}'" for key in inputs)
+            self.logger.info("Using inputs %s", names)
+
+        if overrides:
+            names = ", ".join(f"'{key}'" for key in overrides)
+            self.logger.info("Using overrides %s", names)
+
+    @override
+    def run_after_task_grouping(self, *, run_id: str, task_ids: List[str], **future_kwargs):
+        self.logger.info("Dynamic DAG detected; task-based logging is enabled")
+
+    @override
+    def run_after_task_expansion(self, **future_kwargs):
+        pass  # Note currently uses; required for TaskGroupingHook
+
+    @override
+    def run_before_task_submission(
+        self,
+        *,
+        run_id: str,
+        task_id: str,
+        spawning_task_id: Optional[str],
+        **future_kwargs,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id))
+
+        if spawning_task_id:
+            self.logger.debug("Spawning task and submitting to executor")
+        else:
+            self.logger.debug("Initializing new task and submitting to executor")
+
+    @override
+    def run_before_task_execution(
+        self,
+        *,
+        task_id: str,
+        run_id: str,
+        nodes: List[HamiltonNode],
+        **future_kwargs,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id))
+
+        # Do not log if the task matches the inputs or overrides
+        if task_id in self._inputs_nodes or task_id in self._override_nodes:
+            return
+
+        message = "Starting execution"
+        if len(nodes) == 1 and nodes[0].name == task_id:  # single node task
+            self.logger.debug(message)
+        else:
+            message += " of nodes %s"
+            names = ", ".join(f"'{node.name}'" for node in nodes)
+            self.logger.debug(message, names)
+
+    @override
+    def run_before_node_execution(
+        self,
+        *,
+        node_name: str,
+        node_kwargs: Dict[str, Any],
+        task_id: Optional[str],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id, node=node_name))
+
+        message = "Starting execution"
+        extra = {"include_task_node": True}
+        if node_kwargs:
+            message += " with dependencies %s"
+            params = ", ".join(f"'{key}'" for key in node_kwargs)
+            self.logger.debug(message, params, extra=extra)
+        else:
+            message += " without dependencies"
+            self.logger.debug(message, extra=extra)
+
+    @override
+    def run_after_node_execution(
+        self,
+        *,
+        node_name: str,
+        error: Optional[Exception],
+        success: bool,
+        task_id: Optional[str],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Reset context before logging via the token
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id))
+
+        # Logger should use previous context and include node information for this method
+        extra = {
+            "override_context": _LoggingContext(graph=run_id, task=task_id, node=node_name),
+            "node_context": True,
+        }
+
+        if success:
+            log_func = self.logger.debug if task_id else self.logger.info
+            log_func("Finished execution [OK]", extra=extra)
+        elif error:
+            self.logger.exception("Encountered error", extra=extra)
+            self._exception_logged = True
+
+    @override
+    def run_after_task_execution(
+        self,
+        *,
+        task_id: str,
+        run_id: str,
+        success: bool,
+        error: Exception,
+        **future_kwargs,
+    ):
+        # Reset context before logging
+        _local_context.set(_LoggingContext(graph=run_id))
+
+        # Logger should use previous context for this method
+        extra = {"override_context": _LoggingContext(graph=run_id, task=task_id)}
+
+        # Do not log if the task matches the inputs or overrides
+        if task_id in self._inputs_nodes or task_id in self._override_nodes:
+            return
+
+        if success:
+            self.logger.debug("Finished execution [OK]", extra=extra)
+        elif error:
+            self.logger.error("Execution failed due to errors", extra=extra)
+
+    @override
+    def run_after_task_return(
+        self,
+        *,
+        run_id: str,
+        task_id: str,
+        nodes: List[Node],
+        success: bool,
+        error: Optional[Exception],
+        **future_kwargs,
+    ):
+        # Hard reset context before logging
+        _local_context.set(_LoggingContext(graph=run_id))
+
+        # Logger should use previous context for this method
+        extra = {"override_context": _LoggingContext(graph=run_id, task=task_id)}
+
+        if success:
+            # Input and override tasks should be logged as debug
+            if task_id in self._inputs_nodes or task_id in self._override_nodes:
+                log_func = self.logger.debug
+            else:
+                log_func = self.logger.info
+            log_func("Task completed [OK]", extra=extra)
+        elif error and not self._exception_logged:
+            # NOTE: _exception_logged is used to prevent duplicate exception logging
+            self.logger.exception("Task completion failed due to errors", extra=extra)
+            self._exception_logged = True
+
+    @override
+    def run_after_graph_execution(
+        self,
+        *,
+        success: bool,
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Hard reset context before logging
+        _local_context.set(_LoggingContext())
+
+        # Logger should use previous context for this method
+        extra = {"override_context": _LoggingContext(graph=run_id)}
+
+        if success:
+            self.logger.info("Finished graph execution [OK]", extra=extra)
+        else:
+            self.logger.error("Graph execution failed due to errors", extra=extra)
+
+
+class AsyncLoggingAdapter(GraphExecutionHook, BasePreNodeExecute, BasePostNodeExecuteAsync):
+    """Async version of the `LoggingAdapter`.
+
+    This adapter logs the following hamilton events:
+    - Graph start (`GraphExecutionHook`)
+    - Node pre-execution (`BasePreNodeExecute`)
+    - Node post-execution (`BasePostNodeExecuteAsync`)
+    - Graph completion  (`GraphExecutionHook`)
+
+    Note that this adapter is intended to be used with the async driver. Due to current limitations
+    with the async driver, this adapter is only able to approximate when the async node has been
+    submitted. It cannot currently log the exact moment the async node begins execution.
+    """
+
+    def __init__(self, logger: Union[str, logging.Logger, None] = None) -> None:
+        self._impl = LoggingAdapter(logger)
+
+    @override
+    def run_before_graph_execution(
+        self,
+        *,
+        inputs: Dict[str, Any],
+        overrides: Dict[str, Any],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        self._impl.run_before_graph_execution(inputs=inputs, overrides=overrides, run_id=run_id)
+
+    @override
+    def pre_node_execute(
+        self, *, run_id: str, node_: Node, kwargs: Dict[str, Any], task_id: Optional[str] = None
+    ):
+        # NOTE: We call the base synchronous method here in order to approximate when the async task
+        # has bee submitted. This is a workaround until further work is done on the async adapter.
+
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=None, node=node_.name))
+
+        message = "Submitting async node"
+        extra = {"include_task_node": True}
+        if kwargs:
+            message += " with dependencies %s"
+            params = ", ".join(f"'{key}'" for key in kwargs)
+            self._impl.logger.debug(message, params, extra=extra)
+        else:
+            message += " without dependencies"
+            self._impl.logger.debug(message, extra=extra)
+
+    @override
+    async def post_node_execute(
+        self,
+        *,
+        run_id: str,
+        node_: Node,
+        kwargs: Dict[str, Any],
+        success: bool,
+        error: Optional[Exception],
+        result: Any,
+        task_id: Optional[str] = None,
+    ):
+        self._impl.run_after_node_execution(
+            node_name=node_.name,
+            error=error,
+            success=success,
+            task_id=task_id,
+            run_id=run_id,
+        )
+
+    @override
+    def run_after_graph_execution(
+        self,
+        *,
+        success: bool,
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        self._impl.run_after_graph_execution(success=success, run_id=run_id)
diff --git a/tests/plugins/test_logging.py b/tests/plugins/test_logging.py
new file mode 100644
index 000000000..f1ce4e34a
--- /dev/null
+++ b/tests/plugins/test_logging.py
@@ -0,0 +1,374 @@
+import asyncio
+import logging
+
+import pytest
+
+from hamilton import ad_hoc_utils, async_driver, driver
+from hamilton.execution import executors
+
+# from hamilton.plugins.h_dask import DaskExecutor   # FIXME: Not available CI (see below)
+from hamilton.plugins.h_logging import AsyncLoggingAdapter, LoggingAdapter, get_logger
+
+# from hamilton.plugins.h_ray import RayTaskExecutor  # FIXME: Not available in the CI (see below)
+from hamilton.plugins.h_threadpool import FutureAdapter
+
+from . import test_logging_task_nodes
+
+
+def _split_log_messages(caplog, name):
+    debug, info, warning, error = [], [], [], []
+    for record in caplog.records:
+        if record.name == name:
+            if record.levelno == logging.DEBUG:
+                debug.append(record.message)
+            elif record.levelno == logging.INFO:
+                info.append(record.message)
+            elif record.levelno == logging.WARNING:
+                warning.append(record.message)
+            elif record.levelno == logging.ERROR:
+                error.append(record.message)
+    return debug, info, warning, error
+
+
+def test_logging_serial_nodes_at_info_level(caplog):
+    """Test logging of serial nodes at INFO level - log order matters for this test."""
+
+    name = "test_logging_serial_nodes_at_info_level"
+    caplog.set_level(logging.INFO, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b(a: str) -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return a + " b"
+
+    def c(b: str) -> str:
+        return b + " c"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+    result = dr.execute(["c"])
+    assert result["c"] == "a b c"
+
+    messages = [record.message for record in caplog.records if record.name == name]
+
+    assert messages[0].startswith("Graph run")
+    assert messages[1:-1] == [
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Context aware message",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+    ]
+    assert messages[-1].endswith("- Finished graph execution [OK]")
+
+    levels = [record.levelname for record in caplog.records if record.name == name]
+    assert levels == ["INFO", "INFO", "WARNING", "INFO", "INFO", "INFO"]
+
+
+def test_logging_serial_nodes_at_debug_level(caplog):
+    """Test logging of serial nodes at DEBUG level - log order matters for this test."""
+
+    name = "test_logging_serial_nodes_at_debug_level"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b(a: str) -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return a + " b"
+
+    def c(b: str) -> str:
+        return b + " c"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+
+    result = dr.execute(["c"])
+    assert result["c"] == "a b c"
+
+    messages = [record.message for record in caplog.records if record.name == name]
+    assert messages[0].startswith("Graph run")
+    assert messages[1:-1] == [
+        "Node 'a' - Starting execution without dependencies",
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Starting execution with dependencies 'a'",
+        "Node 'b' - Context aware message",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Starting execution with dependencies 'b'",
+        "Node 'c' - Finished execution [OK]",
+    ]
+    assert messages[-1].endswith("- Finished graph execution [OK]")
+
+    levels = [record.levelname for record in caplog.records if record.name == name]
+    assert levels == ["INFO", "DEBUG", "INFO", "DEBUG", "WARNING", "INFO", "DEBUG", "INFO", "INFO"]
+
+
+@pytest.mark.parametrize("adapter", [None, FutureAdapter])
+def test_logging_branching_nodes(caplog, adapter):
+    """Test logging of branching nodes at multiple logging levels."""
+
+    name = "test_logging_branching_nodes"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b() -> str:
+        return "b"
+
+    def c() -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return "c"
+
+    def d(a: str, b: str) -> str:
+        return a + " " + b + " d"
+
+    def e(c: str) -> str:
+        return c + " e"
+
+    def f(d: str, e: str) -> str:
+        return d + " " + e + " f"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c, d, e, f)
+    adapters = [LoggingAdapter(name)]
+    if adapter:
+        adapters.append(adapter())
+    dr = driver.Builder().with_modules(modules).with_adapters(*adapters).build()
+
+    result = dr.execute(["f"])
+    assert result["f"] == "a b d c e f"
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert set(info[1:-1]) == {
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+        "Node 'd' - Finished execution [OK]",
+        "Node 'e' - Finished execution [OK]",
+        "Node 'f' - Finished execution [OK]",
+    }
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(debug) == {
+        "Node 'a' - Starting execution without dependencies",
+        "Node 'b' - Starting execution without dependencies",
+        "Node 'c' - Starting execution without dependencies",
+        "Node 'd' - Starting execution with dependencies 'a', 'b'",
+        "Node 'e' - Starting execution with dependencies 'c'",
+        "Node 'f' - Starting execution with dependencies 'd', 'e'",
+    }
+
+    assert len(warning) == 1
+    assert warning[0] == "Node 'c' - Context aware message"
+
+
+def test_logging_async_nodes(caplog):
+    """Test logging of async nodes at multiple logging levels."""
+
+    name = "test_logging_async_nodes"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    async def a() -> str:
+        return "a"
+
+    async def b() -> str:
+        return "b"
+
+    async def c() -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return "c"
+
+    async def d(a: str, b: str) -> str:
+        return a + " " + b + " d"
+
+    async def e(c: str) -> str:
+        return c + " e"
+
+    async def f(d: str, e: str) -> str:
+        return d + " " + e + " f"
+
+    async def run_async(module, name):
+        dr = (
+            await async_driver.Builder()  # type: ignore
+            .with_modules(module)
+            .with_adapters(AsyncLoggingAdapter(name))
+            .build()
+        )
+        result = await dr.execute(["f"])
+        return result
+
+    module = ad_hoc_utils.create_temporary_module(a, b, c, d, e, f)
+    result = asyncio.run(run_async(module, name))
+
+    assert result["f"] == "a b d c e f"
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert set(info[1:-1]) == {
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+        "Node 'd' - Finished execution [OK]",
+        "Node 'e' - Finished execution [OK]",
+        "Node 'f' - Finished execution [OK]",
+    }
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(debug) == {
+        "Node 'a' - Submitting async node without dependencies",
+        "Node 'b' - Submitting async node without dependencies",
+        "Node 'c' - Submitting async node without dependencies",
+        "Node 'd' - Submitting async node with dependencies 'a', 'b'",
+        "Node 'e' - Submitting async node with dependencies 'c'",
+        "Node 'f' - Submitting async node with dependencies 'd', 'e'",
+    }
+
+    assert len(warning) == 1
+    assert warning[0] == "Node 'c' - Context aware message"
+
+
+@pytest.mark.parametrize(
+    ["executor_type", "executor_args", "check_context"],
+    [
+        (executors.SynchronousLocalTaskExecutor, {}, True),
+        (executors.MultiProcessingExecutor, {"max_tasks": 1}, False),
+        (executors.MultiThreadingExecutor, {"max_tasks": 2}, True),
+        # (RayTaskExecutor, {}, True),  # FIXME: Not available in the CI environment
+        # (DaskExecutor, {"client": None}, False),  # FIXME: Not available in the CI environment
+    ],
+)
+def test_logging_parallel_nodes(caplog, executor_type, executor_args, check_context):
+    """Test logging of parallel nodes at multiple logging levels."""
+
+    # NOTE: These test is brittle, as it depends on undocumented names of the expanded tasks.
+
+    name = "test_logging_parallel_nodes"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    # FIXME: dask is not available in the CI environment
+    # if executor_type == DaskExecutor:
+    #     import dask.distributed
+
+    #     cluster = dask.distributed.LocalCluster(n_workers=5)
+    #     client = dask.distributed.Client(cluster)
+    #     executor_args["client"] = client
+
+    adapters = [LoggingAdapter(name)]
+    dr = (
+        driver.Builder()
+        .with_modules(test_logging_task_nodes)
+        .with_adapters(*adapters)
+        .enable_dynamic_execution(allow_experimental_mode=True)
+        .with_remote_executor(executor_type(**executor_args))
+        .build()
+    )
+
+    result = dr.execute(["f"])
+    assert result["f"] == 20
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert info[1].endswith("task-based logging is enabled")
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(info[2:-1]) == {
+        "Task 'b' - Task completed [OK]",
+        "Task 'expand-c' - Task completed [OK]",
+        "Task 'expand-c.0.block-c' - Task completed [OK]",
+        "Task 'expand-c.1.block-c' - Task completed [OK]",
+        "Task 'expand-c.2.block-c' - Task completed [OK]",
+        "Task 'expand-c.3.block-c' - Task completed [OK]",
+        "Task 'expand-c.4.block-c' - Task completed [OK]",
+        "Task 'collect-c' - Task completed [OK]",
+        "Task 'f' - Task completed [OK]",
+    }
+
+    # Note: Certain executors do not log task and node level debug messages (especially if they
+    # are not running in the same process as the driver).
+    local_debug_log = {
+        "Task 'b' - Initializing new task and submitting to executor",
+        "Task 'b' - Starting execution",
+        "Task 'b' - Starting execution without dependencies",
+        "Task 'b' - Node 'b' - Finished execution [OK]",
+        "Task 'b' - Finished execution [OK]",
+        "Task 'expand-c' - Initializing new task and submitting to executor",
+        "Task 'expand-c' - Starting execution of nodes 'c'",
+        "Task 'expand-c' - Starting execution with dependencies 'b'",
+        "Task 'expand-c' - Node 'c' - Finished execution [OK]",
+        "Task 'expand-c' - Finished execution [OK]",
+        "Task 'expand-c.0.block-c' - Spawning task and submitting to executor",
+        "Task 'expand-c.1.block-c' - Spawning task and submitting to executor",
+        "Task 'expand-c.2.block-c' - Spawning task and submitting to executor",
+        "Task 'expand-c.3.block-c' - Spawning task and submitting to executor",
+        "Task 'expand-c.4.block-c' - Spawning task and submitting to executor",
+        "Task 'collect-c' - Initializing new task and submitting to executor",
+        "Task 'collect-c' - Starting execution of nodes 'e'",
+        "Task 'collect-c' - Starting execution with dependencies 'd'",
+        "Task 'collect-c' - Node 'e' - Finished execution [OK]",
+        "Task 'collect-c' - Finished execution [OK]",
+        "Task 'f' - Initializing new task and submitting to executor",
+        "Task 'f' - Starting execution",
+        "Task 'f' - Starting execution with dependencies 'e'",
+        "Task 'f' - Node 'f' - Finished execution [OK]",
+        "Task 'f' - Finished execution [OK]",
+    }
+    assert local_debug_log.difference(set(debug)) == set()
+
+    if check_context:
+        assert set(warning) == {
+            "Task 'expand-c.0.block-c' - Context aware message",
+            "Task 'expand-c.1.block-c' - Context aware message",
+            "Task 'expand-c.2.block-c' - Context aware message",
+            "Task 'expand-c.3.block-c' - Context aware message",
+            "Task 'expand-c.4.block-c' - Context aware message",
+        }
+
+
+def test_logging_with_inputs(caplog):
+    """Test logging of nodes with inputs."""
+
+    name = "test_logging_with_inputs"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a(x: str) -> str:
+        return x
+
+    modules = ad_hoc_utils.create_temporary_module(a)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+
+    result = dr.execute(["a"], inputs={"x": "test"})
+    assert result["a"] == "test"
+
+    _, info, _, _ = _split_log_messages(caplog, name)
+
+    assert info[1].endswith("Using inputs 'x'")
+
+
+def test_logging_with_overrides(caplog):
+    """Test logging of nodes with overrides."""
+
+    name = "test_logging_with_overrides"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a(x: str) -> str:
+        return x
+
+    modules = ad_hoc_utils.create_temporary_module(a)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+
+    result = dr.execute(["a"], overrides={"a": "test"})
+    assert result["a"] == "test"
+
+    _, info, _, _ = _split_log_messages(caplog, name)
+
+    assert info[1].endswith("Using overrides 'a'")
diff --git a/tests/plugins/test_logging_task_nodes.py b/tests/plugins/test_logging_task_nodes.py
new file mode 100644
index 000000000..323ff5300
--- /dev/null
+++ b/tests/plugins/test_logging_task_nodes.py
@@ -0,0 +1,29 @@
+# NOTE: This file contains nodes for the test 'test_logging_parallel_nodes' in
+# test_logging_task_nodes.py. They are required to be in a separate file in order to properly
+# test the multi-processing executor.
+
+from hamilton.htypes import Collect, Parallelizable
+from hamilton.plugins.h_logging import get_logger
+
+
+def b() -> int:
+    return 5
+
+
+def c(b: int) -> Parallelizable[int]:
+    for i in range(b):
+        yield i
+
+
+def d(c: int) -> int:
+    logger = get_logger("test_logging_parallel_nodes")
+    logger.warning("Context aware message")
+    return 2 * c
+
+
+def e(d: Collect[int]) -> int:
+    return sum(d)
+
+
+def f(e: int) -> int:
+    return e


Reviews:
user: login: ellipsis-dev[bot]
body: :x: Changes requested. Reviewed everything up to aaef55b679568e23ba4bb54937ea7a1781ef09dd in <ellipsistime>3 minutes and 0 seconds</ellipsistime>

<details>
<summary>More details</summary>

 - Looked at `1617` lines of code in `12` files
 - Skipped `0` files when reviewing.
 - Skipped posting `22` drafted comments based on config settings.


<details>
<summary>1. <code>hamilton/plugins/h_logging.py:81</code></summary>

- **Draft comment:** 
Consider adding a comment on how the extra fields (override_context, skip_context, node_context) affect the log message prefix for developers.
- **Reason this comment was not posted:** 
Confidence changes required: `20%` <= threshold `50%`
None

</details>

<details>
<summary>2. <code>hamilton/plugins/h_logging.py:143</code></summary>

- **Draft comment:** 
Document the logging adapter's handling of input/override nodes in run_before_task_execution. Clarify why logging is skipped for tasks matching these sets.
- **Reason this comment was not posted:** 
Confidence changes required: `20%` <= threshold `50%`
None

</details>

<details>
<summary>3. <code>hamilton/plugins/h_logging.py:309</code></summary>

- **Draft comment:** 
Ensure that the signature of run_after_task_resolution matches interface expectations. Although the 'result' parameter is omitted here, consider documenting that decision.
- **Reason this comment was not posted:** 
Confidence changes required: `30%` <= threshold `50%`
None

</details>

<details>
<summary>4. <code>tests/plugins/test_logging.py:145</code></summary>

- **Draft comment:** 
Use helper functions or inline comments to clarify the brittle nature of tests that depend on task naming conventions.
- **Reason this comment was not posted:** 
Confidence changes required: `20%` <= threshold `50%`
None

</details>

<details>
<summary>5. <code>tests/plugins/test_logging.py:351</code></summary>

- **Draft comment:** 
Consider adding a comment clarifying the expected behavior of the log when using overrides, ensuring consistency with the LoggingAdapter’s logic.
- **Reason this comment was not posted:** 
Confidence changes required: `10%` <= threshold `50%`
None

</details>

<details>
<summary>6. <code>tests/lifecycle/lifecycle_adapters_for_testing.py:54</code></summary>

- **Draft comment:** 
The global 'sequence_number' used in ExtendToTrackCalls may not be safe in concurrent or parallel test environments. Consider using an instance-level counter or a thread‐safe mechanism.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>7. <code>tests/lifecycle/test_lifecycle_adapters_end_to_end_task_based.py:130</code></summary>

- **Draft comment:** 
Magic numbers in assertions (e.g. '2 + 5 + 1 + 1') reduce clarity. Consider defining named constants or adding comments to explain the expected counts for each task group.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>8. <code>tests/plugins/test_logging.py:320</code></summary>

- **Draft comment:** 
The test 'test_logging_parallel_nodes' uses a hardcoded set of debug log messages in 'local_debug_log'. This brittle check may fail if internal log message formats change. Consider using pattern‐based matching or more flexible assertions.
- **Reason this comment was not posted:** 
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50%
While the comment raises a valid point about test brittleness, this appears to be an intentional design choice. The test is specifically checking the exact format of log messages across different executors. The test author is aware of this, as evidenced by their comment. Pattern matching could make the test less precise, which may defeat its purpose of catching unintended changes to log formats.
The comment suggests a reasonable alternative approach with pattern matching. This could make the tests more maintainable if log formats need to change frequently.
However, exact string matching serves a purpose here - it acts as a safeguard against unintended changes to log formats, which could be important for users relying on these formats.
The comment should be deleted since the test's brittleness is intentional and already acknowledged in the code comments. The exact string matching appears to be a deliberate choice to catch log format changes.

</details>

<details>
<summary>9. <code>tests/plugins/test_logging.py:370</code></summary>

- **Draft comment:** 
The test 'test_logging_with_overrides' asserts that the info log message ends with a specific string ('Using overrides 'a''). Relying on exact string comparisons for log messages can be brittle. Consider asserting for a substring or using regex-based matching to allow flexibility in log formatting.
- **Reason this comment was not posted:** 
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
Since this is a test specifically for the logging functionality, being precise about the log message format is actually important. The test is verifying the exact format of log messages produced by the logging adapter. Making the test more flexible could hide changes to the log format that we want to catch. Additionally, this test follows the same pattern as other logging tests in the file that use endswith() checks.
The comment has a valid point about test brittleness in general - exact string matching can make tests fragile. However, in this specific case, we're testing a logging system where the exact format matters.
While reducing test brittleness is generally good, in this case the precise log format is part of the API we're testing. The existing approach using endswith() is consistent with other tests in the file and appropriate for testing logging output.
The comment should be deleted. The current test approach using endswith() is appropriate for testing logging functionality where the exact message format matters.

</details>

<details>
<summary>10. <code>hamilton/execution/executors.py:24</code></summary>

- **Draft comment:** 
The docstring for TaskFuture currently says 'This a clean wrapper over a python future...'. It would be clearer to say 'This is a clean wrapper over a Python future...'.
- **Reason this comment was not posted:** 
Comment was on unchanged code.

</details>

<details>
<summary>11. <code>hamilton/execution/executors.py:102</code></summary>

- **Draft comment:** 
In the docstring of base_execute_task, 'a diciontary of the results' should be corrected to 'a dictionary of the results'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>12. <code>hamilton/execution/grouping.py:176</code></summary>

- **Draft comment:** 
Typo found: 'betweeen' should be corrected to 'between'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>13. <code>hamilton/execution/grouping.py:203</code></summary>

- **Draft comment:** 
Typo found: 'Fina all the Parallelizable[] nodes in the DAG' should be 'Find all the Parallelizable[] nodes in the DAG'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>14. <code>hamilton/execution/grouping.py:233</code></summary>

- **Draft comment:** 
Typo found in comment: 'In thie case of a strange override' should be 'In this case of a strange override'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>15. <code>hamilton/execution/grouping.py:392</code></summary>

- **Draft comment:** 
Typo found: 'guarenteed topological order' should be 'guaranteed topological order'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>16. <code>hamilton/lifecycle/api.py:472</code></summary>

- **Draft comment:** 
Typo: In the docstring for 'run_after_task_resolution', change 'executor,on' to 'executor, on' (add a space after the comma) for better readability.
- **Reason this comment was not posted:** 
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
While the comment is technically correct about a missing space, the review rules emphasize not making purely informative comments or obvious/unimportant changes. A missing space in a docstring is a very minor formatting issue that doesn't impact code functionality or readability significantly.
The typo does slightly impact docstring readability. Documentation quality is important for developer experience.
While documentation quality matters, this is an extremely minor formatting issue that doesn't significantly impact understanding. The rules specifically say not to make unimportant comments.
The comment should be deleted as it points out an extremely minor formatting issue that doesn't meaningfully impact code quality or understanding.

</details>

<details>
<summary>17. <code>hamilton/lifecycle/base.py:287</code></summary>

- **Draft comment:** 
Typo in the docstring of BaseDoValidateInput: change 'maches' to 'matches'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>18. <code>hamilton/plugins/h_logging.py:389</code></summary>

- **Draft comment:** 
Typo: Change 'has bee submitted' to 'has been submitted' in the asynchronous adapter comment.
- **Reason this comment was not posted:** 
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
While this is technically a typo, it's in a comment and doesn't affect functionality. The meaning is still clear despite the typo. The rules emphasize focusing on actual code changes and avoiding trivial changes. Fixing comment typos is not one of the priority rules listed.
The typo does make the code slightly less professional. Documentation quality is important for maintainability.
However, this is an extremely minor issue in an internal comment, not user-facing documentation. The meaning is still perfectly clear. Opening a PR just to fix this would create more overhead than value.
This comment should be deleted as it points out an extremely minor typo in a comment that doesn't impact code functionality or clarity.

</details>

<details>
<summary>19. <code>hamilton/plugins/h_logging.py:128</code></summary>

- **Draft comment:** 
Typographical issue: Remove the extra closing parenthesis in the bullet list items for 'Task pre-execution (`TaskExecutionHook`))' and 'Task post-execution (`TaskExecutionHook`))'. The entries should have matching parentheses.
- **Reason this comment was not posted:** 
Marked as duplicate.

</details>

<details>
<summary>20. <code>tests/lifecycle/lifecycle_adapters_for_testing.py:56</code></summary>

- **Draft comment:** 
Typographical issue: In the comment on line 56, consider changing "cause" to "because" and "its" to "it's" for clarity.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>21. <code>tests/lifecycle/test_lifecycle_adapters_end_to_end_task_based.py:154</code></summary>

- **Draft comment:** 
Typographical error: In the comment for the task_ids assertion, 'at lease one for the inner task that failed' should be 'at least one for the inner task that failed'.
- **Reason this comment was not posted:** 
Comment was not on a location in the diff, so it can't be submitted as a review comment.

</details>

<details>
<summary>22. <code>tests/plugins/test_logging.py:256</code></summary>

- **Draft comment:** 
Typo in comment: "NOTE: These test is brittle..." should probably be "NOTE: This test is brittle..." or "These tests are brittle..." for proper grammar.
- **Reason this comment was not posted:** 
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50%
While this is technically correct, our rules say to only keep comments that require code changes. Grammar fixes in comments are very low value and don't affect functionality. The comment is still understandable despite the minor grammar error.
The grammar error could make the codebase look less professional. Poor grammar in comments could set a bad precedent.
While professionalism matters, this is a very minor issue in a test file. The meaning is still clear and fixing grammar in comments is not a high priority use of review time.
Delete this comment as it only addresses grammar in a comment rather than suggesting any meaningful code changes.

</details>


Workflow ID: <workflowid>`wflow_3QRUI1P2obJZcqsP`</workflowid>

</details>


----
**Want Ellipsis to fix these issues?** Tag `@ellipsis-dev` in a comment. You can customize Ellipsis with :+1: / :-1: [feedback](https://docs.ellipsis.dev/review), review rules, user-specific overrides, `quiet` mode, and [more](https://docs.ellipsis.dev/config).

state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T04:22:59Z
user: login: elijahbenizzy
body: Looks good, very little blast radius. Let's get #1287 merged first then merge this.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T03:25:54Z
user: login: cswartzvi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T23:54:51Z
user: login: elijahbenizzy
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T03:57:30Z
user: login: elijahbenizzy
body: Looks good -- once we merge the other we can merge this! Will likely want to rebase off of main after that due to the squash and merge.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T03:59:00Z

Review comments:
diff_hunk: @@ -0,0 +1,432 @@
+"""Synchronous/asynchronous adapter and functions for context-aware logging with Hamilton."""
+
+import logging
+import sys
+from contextvars import ContextVar
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any, Dict, List, Mapping, MutableMapping, Optional, Set, Union
+
+from hamilton.graph_types import HamiltonNode
+from hamilton.lifecycle.api import (
+    GraphExecutionHook,
+    NodeExecutionHook,
+    TaskExecutionHook,
+    TaskGroupingHook,
+    TaskResolutionHook,
+    TaskSubmissionHook,
+)
+from hamilton.lifecycle.base import BasePostNodeExecuteAsync, BasePreNodeExecute
+from hamilton.node import Node
+
+try:
+    from typing import override
+except ImportError:
+    override = lambda x: x  # noqa E731
+
+
+if sys.version_info >= (3, 12):
+    LoggerAdapter = logging.LoggerAdapter[logging.Logger]
+else:
+    if TYPE_CHECKING:
+        LoggerAdapter = logging.LoggerAdapter[logging.Logger]
+    else:
+        LoggerAdapter = logging.LoggerAdapter
+
+
+@dataclass(frozen=True)
+class _LoggingContext:
+    """Represents the current logging context."""
+
+    graph: Optional[str] = None
+    node: Optional[str] = None
+    task: Optional[str] = None
+
+
+# Context variables for context-aware logging
+_local_context = ContextVar("context", default=_LoggingContext())
+
+
+def get_logger(name: Optional[str] = None) -> "ContextLogger":
+    """Returns a context-aware logger for the specified name (created if necessary).
+
+    :param name: Name of the logger, defaults to root logger if not provided.
+    """
+    logger = logging.getLogger(name)
+    return ContextLogger(logger)
+
+
+class ContextLogger(LoggerAdapter):
+    """Custom logger adapter for Hamilton that adds context to log messages.
+
+    This logger a adds context-aware prefix to log messages based on the current execution. The
+    logger is intended to be used with hamilton the `LoggingAdapter` lifecycle adapter. The context
+    is both thread-safe and async-safe. Context includes the following details:
+    - Graph run
+    - Task ID
+    - Node ID
+
+    The adapter also supports the following extra fields:
+    - `override_context`: Overrides the current context with the specified context.
+    - `skip_context`: Skips the context for the current log message.
+    - `node_context`: Includes additional node information for task-based log messages.
+    """
+
+    @override
+    def process(
+        self, msg: str, kwargs: MutableMapping[str, Any]
+    ) -> tuple[str, MutableMapping[str, Any]]:
+        # Ensure that the extra fields are passed through correctly
+        kwargs["extra"] = {**(self.extra or {}), **(kwargs.get("extra") or {})}
+
+        # Add the current prefix to the log message
+        prefix = self._get_current_context(kwargs["extra"])
+        msg = f"{prefix}{msg}"
+
+        return (msg, kwargs)
+
+    def _get_current_context(self, extra: Mapping[str, Any]) -> str:
+        """Returns the current context."""
+
+        # Extra option to override context
+        context = extra.get("override_context", None)
+        if not isinstance(context, _LoggingContext):
+            context = _local_context.get()
+
+        # Extra option to skip context
+        if "skip_context" in extra:
+            return ""
+
+        if context.task:
+            # Extra option to include node information on task-based log messages
+            if context.node and "node_context" in extra:
+                return f"Task '{context.task}' - Node '{context.node}' - "
+            return f"Task '{context.task}' - "
+
+        if context.node:
+            return f"Node '{context.node}' - "
+
+        if context.graph:
+            return f"Graph run '{context.graph}' - "
+
+        return ""
+
+
+class LoggingAdapter(
+    GraphExecutionHook,
+    NodeExecutionHook,
+    TaskGroupingHook,
+    TaskSubmissionHook,
+    TaskExecutionHook,
+    TaskResolutionHook,
+):
+    """Hamilton lifecycle adapter that logs runtime execution events.
+
+    This adapter logs the following hamilton events:
+    - Graph start (`GraphExecutionHook`)
+    - Task grouping (`TaskGroupingHook`)
+    - Task submission (`TaskSubmissionHook`)
+    - Task pre-execution (`TaskExecutionHook`))
+    - Node pre-execution (`NodeExecutionHook`)
+    - Node post-execution (`NodeExecutionHook`)
+    - Task post-execution (`TaskExecutionHook`))
+    - Task resolution  (`TaskResolutionHook`)
+    - Graph completion  (`GraphExecutionHook`)
+
+    This adapter can be run with both node-based and task-based execution (using the V2 executor).
+    When run with a node-based executor, the adapter logs the execution of each *node* as `INFO`.
+    When run with a task-based executor, the adapter logs the execution of each *task* as `INFO`
+    and the execution of each *node* as `DEBUG`.
+    """
+
+    def __init__(self, logger: Union[str, logging.Logger, None] = None) -> None:
+        # Precompute or overridden nodes
+        self._inputs_nodes: Set[str] = set()
+        self._override_nodes: Set[str] = set()
+
+        if logger is None:
+            self.logger = logging.getLogger(__name__)
+        elif isinstance(logger, logging.Logger):
+            self.logger = logger
+        else:
+            self.logger = logging.getLogger(logger)
+
+        if not isinstance(self.logger, ContextLogger):
+            self.logger = ContextLogger(self.logger)
+
+        self._exception_logged = False  # For tracking remote exceptions
+
+    @override
+    def run_before_graph_execution(
+        self,
+        *,
+        inputs: Optional[Dict[str, Any]],
+        overrides: Optional[Dict[str, Any]],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id))
+
+        self._inputs_nodes.update(inputs or [])
+        self._override_nodes.update(overrides or [])
+
+        message = "Starting graph execution"
+        self.logger.info(message)
+
+        if inputs:
+            names = ", ".join(f"'{key}'" for key in inputs)
+            self.logger.info("Using inputs %s", names)
+
+        if overrides:
+            names = ", ".join(f"'{key}'" for key in overrides)
+            self.logger.info("Using overrides %s", names)
+
+    @override
+    def run_after_task_grouping(self, *, run_id: str, task_ids: List[str], **future_kwargs):
+        self.logger.info("Dynamic DAG detected; task-based logging is enabled")
+
+    @override
+    def run_after_task_expansion(self, **future_kwargs):
+        pass  # Note currently uses; required for TaskGroupingHook
+
+    @override
+    def run_before_task_submission(
+        self,
+        *,
+        run_id: str,
+        task_id: str,
+        spawning_task_id: Optional[str],
+        **future_kwargs,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id))
+
+        if spawning_task_id:
+            self.logger.debug("Spawning task and submitting to executor")
+        else:
+            self.logger.debug("Initializing new task and submitting to executor")
+
+    @override
+    def run_before_task_execution(
+        self,
+        *,
+        task_id: str,
+        run_id: str,
+        nodes: List[HamiltonNode],
+        **future_kwargs,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id))
+
+        # Do not log if the task matches the inputs or overrides
+        if task_id in self._inputs_nodes or task_id in self._override_nodes:
+            return
+
+        message = "Starting execution"
+        if len(nodes) == 1 and nodes[0].name == task_id:  # single node task
+            self.logger.debug(message)
+        else:
+            message += " of nodes %s"
+            names = ", ".join(f"'{node.name}'" for node in nodes)
+            self.logger.debug(message, names)
+
+    @override
+    def run_before_node_execution(
+        self,
+        *,
+        node_name: str,
+        node_kwargs: Dict[str, Any],
+        task_id: Optional[str],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Set context before logging
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id, node=node_name))
+
+        message = "Starting execution"
+        extra = {"include_task_node": True}
+        if node_kwargs:
+            message += " with dependencies %s"
+            params = ", ".join(f"'{key}'" for key in node_kwargs)
+            self.logger.debug(message, params, extra=extra)
+        else:
+            message += " without dependencies"
+            self.logger.debug(message, extra=extra)
+
+    @override
+    def run_after_node_execution(
+        self,
+        *,
+        node_name: str,
+        error: Optional[Exception],
+        success: bool,
+        task_id: Optional[str],
+        run_id: str,
+        **future_kwargs: Any,
+    ):
+        # Reset context before logging via the token
+        _local_context.set(_LoggingContext(graph=run_id, task=task_id))
+
+        # Logger should use previous context and include node information for this method
+        extra = {
+            "override_context": _LoggingContext(graph=run_id, task=task_id, node=node_name),
+            "node_context": True,
+        }
+
+        if success:
+            log_func = self.logger.debug if task_id else self.logger.info
+            log_func("Finished execution [OK]", extra=extra)
+        elif error:
+            self.logger.exception("Encountered error", extra=extra)
+            self._exception_logged = True
+
+    @override
+    def run_after_task_execution(
+        self,
+        *,
+        task_id: str,
+        run_id: str,
+        success: bool,
+        error: Exception,
+        **future_kwargs,
+    ):
+        # Reset context before logging
+        _local_context.set(_LoggingContext(graph=run_id))
+
+        # Logger should use previous context for this method
+        extra = {"override_context": _LoggingContext(graph=run_id, task=task_id)}
+
+        # Do not log if the task matches the inputs or overrides
+        if task_id in self._inputs_nodes or task_id in self._override_nodes:
+            return
+
+        if success:
+            self.logger.debug("Finished execution [Ok]", extra=extra)
+        elif error:
+            self.logger.error("Execution failed due to errors", extra=extra)
+
+    @override
+    def run_after_task_resolution(
+        self,
+        *,
+        run_id: str,
+        task_id: str,
+        nodes: List[Node],
+        success: bool,
+        error: Optional[Exception],
+        **future_kwargs,
+    ):
+        # Hard reset context before logging
+        _local_context.set(_LoggingContext(graph=run_id))
+
+        # Logger should use previous context for this method
+        extra = {"override_context": _LoggingContext(graph=run_id, task=task_id)}
+
+        if success:
+            # Input and override tasks should be logged as debug
+            if task_id in self._inputs_nodes or task_id in self._override_nodes:
+                log_func = self.logger.debug
+            else:
+                log_func = self.logger.info
+            log_func("Task completed [OK]", extra=extra)
+        elif error and not self._exception_logged:
path: hamilton/plugins/h_logging.py
user: login: ellipsis-dev[bot]
body: In `run_after_task_resolution`, the use of `_exception_logged` prevents logging multiple errors. Please document the rationale or consider resetting it per task if multiple failures should each be logged.
created_at: 2025-03-17T04:22:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -651,7 +764,9 @@ def post_task_expand(self, *, run_id: str, task_id: str, parameters: Dict[str, A
 
     @abc.abstractmethod
     def run_after_task_grouping(self, *, run_id: str, task_ids: List[str], **future_kwargs):
-        """Hook that is called after task grouping.
+        """Runs after task grouping. This allows you to capture information about the which tasks were
path: hamilton/lifecycle/api.py
user: login: ellipsis-dev[bot]
body: Typo: In the docstring for `run_after_task_grouping`, consider removing the extra 'the' in the phrase 'capture information about the which tasks were created' to read 'capture information about which tasks were created'.
```suggestion
        """Runs after task grouping. This allows you to capture information about which tasks were
```
created_at: 2025-03-17T04:22:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,432 @@
+"""Synchronous/asynchronous adapter and functions for context-aware logging with Hamilton."""
+
+import logging
+import sys
+from contextvars import ContextVar
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any, Dict, List, Mapping, MutableMapping, Optional, Set, Union
+
+from hamilton.graph_types import HamiltonNode
+from hamilton.lifecycle.api import (
+    GraphExecutionHook,
+    NodeExecutionHook,
+    TaskExecutionHook,
+    TaskGroupingHook,
+    TaskResolutionHook,
+    TaskSubmissionHook,
+)
+from hamilton.lifecycle.base import BasePostNodeExecuteAsync, BasePreNodeExecute
+from hamilton.node import Node
+
+try:
+    from typing import override
+except ImportError:
+    override = lambda x: x  # noqa E731
+
+
+if sys.version_info >= (3, 12):
+    LoggerAdapter = logging.LoggerAdapter[logging.Logger]
+else:
+    if TYPE_CHECKING:
+        LoggerAdapter = logging.LoggerAdapter[logging.Logger]
+    else:
+        LoggerAdapter = logging.LoggerAdapter
+
+
+@dataclass(frozen=True)
+class _LoggingContext:
+    """Represents the current logging context."""
+
+    graph: Optional[str] = None
+    node: Optional[str] = None
+    task: Optional[str] = None
+
+
+# Context variables for context-aware logging
+_local_context = ContextVar("context", default=_LoggingContext())
+
+
+def get_logger(name: Optional[str] = None) -> "ContextLogger":
+    """Returns a context-aware logger for the specified name (created if necessary).
+
+    :param name: Name of the logger, defaults to root logger if not provided.
+    """
+    logger = logging.getLogger(name)
+    return ContextLogger(logger)
+
+
+class ContextLogger(LoggerAdapter):
+    """Custom logger adapter for Hamilton that adds context to log messages.
+
+    This logger a adds context-aware prefix to log messages based on the current execution. The
path: hamilton/plugins/h_logging.py
user: login: ellipsis-dev[bot]
body: Typo: Remove the extra 'a' so that the sentence reads `This logger adds context-aware prefix to log messages.`
```suggestion
    This logger adds context-aware prefix to log messages based on the current execution. The
```
created_at: 2025-03-17T04:22:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,387 @@
+import asyncio
+import logging
+import sys
+
+import pytest
+
+from hamilton import ad_hoc_utils, async_driver, driver
+from hamilton.execution import executors
+from hamilton.htypes import Collect, Parallelizable
+from hamilton.plugins.h_dask import DaskExecutor
+from hamilton.plugins.h_logging import AsyncLoggingAdapter, LoggingAdapter, get_logger
+from hamilton.plugins.h_ray import RayTaskExecutor
+from hamilton.plugins.h_threadpool import FutureAdapter
+
+
+def _split_log_messages(caplog, name):
+    debug, info, warning, error = [], [], [], []
+    for record in caplog.records:
+        if record.name == name:
+            if record.levelno == logging.DEBUG:
+                debug.append(record.message)
+            elif record.levelno == logging.INFO:
+                info.append(record.message)
+            elif record.levelno == logging.WARNING:
+                warning.append(record.message)
+            elif record.levelno == logging.ERROR:
+                error.append(record.message)
+    return debug, info, warning, error
+
+
+def test_logging_serial_nodes_at_info_level(caplog):
+    """Test logging of serial nodes at INFO level - log order matters for this test."""
+
+    name = "test_logging_serial_nodes_at_info_level"
+    caplog.set_level(logging.INFO, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b(a: str) -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return a + " b"
+
+    def c(b: str) -> str:
+        return b + " c"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+    result = dr.execute(["c"])
+    assert result["c"] == "a b c"
+
+    messages = [record.message for record in caplog.records if record.name == name]
+
+    assert messages[0].startswith("Graph run")
+    assert messages[1:-1] == [
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Context aware message",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+    ]
+    assert messages[-1].endswith("- Finished graph execution [OK]")
+
+    levels = [record.levelname for record in caplog.records if record.name == name]
+    assert levels == ["INFO", "INFO", "WARNING", "INFO", "INFO", "INFO"]
+
+
+def test_logging_serial_nodes_at_debug_level(caplog):
+    """Test logging of serial nodes at DEBUG level - log order matters for this test."""
+
+    name = "test_logging_serial_nodes_at_debug_level"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b(a: str) -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return a + " b"
+
+    def c(b: str) -> str:
+        return b + " c"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+
+    result = dr.execute(["c"])
+    assert result["c"] == "a b c"
+
+    messages = [record.message for record in caplog.records if record.name == name]
+    assert messages[0].startswith("Graph run")
+    assert messages[1:-1] == [
+        "Node 'a' - Starting execution without dependencies",
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Starting execution with dependencies 'a'",
+        "Node 'b' - Context aware message",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Starting execution with dependencies 'b'",
+        "Node 'c' - Finished execution [OK]",
+    ]
+    assert messages[-1].endswith("- Finished graph execution [OK]")
+
+    levels = [record.levelname for record in caplog.records if record.name == name]
+    assert levels == ["INFO", "DEBUG", "INFO", "DEBUG", "WARNING", "INFO", "DEBUG", "INFO", "INFO"]
+
+
+@pytest.mark.parametrize("adapter", [None, FutureAdapter])
+def test_logging_branching_nodes(caplog, adapter):
+    """Test logging of branching nodes at multiple logging levels."""
+
+    name = "test_logging_branching_nodes"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b() -> str:
+        return "b"
+
+    def c() -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return "c"
+
+    def d(a: str, b: str) -> str:
+        return a + " " + b + " d"
+
+    def e(c: str) -> str:
+        return c + " e"
+
+    def f(d: str, e: str) -> str:
+        return d + " " + e + " f"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c, d, e, f)
+    adapters = [LoggingAdapter(name)]
+    if adapter:
+        adapters.append(adapter())
+    dr = driver.Builder().with_modules(modules).with_adapters(*adapters).build()
+
+    result = dr.execute(["f"])
+    assert result["f"] == "a b d c e f"
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert set(info[1:-1]) == {
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+        "Node 'd' - Finished execution [OK]",
+        "Node 'e' - Finished execution [OK]",
+        "Node 'f' - Finished execution [OK]",
+    }
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(debug) == {
+        "Node 'a' - Starting execution without dependencies",
+        "Node 'b' - Starting execution without dependencies",
+        "Node 'c' - Starting execution without dependencies",
+        "Node 'd' - Starting execution with dependencies 'a', 'b'",
+        "Node 'e' - Starting execution with dependencies 'c'",
+        "Node 'f' - Starting execution with dependencies 'd', 'e'",
+    }
+
+    assert len(warning) == 1
+    assert warning[0] == "Node 'c' - Context aware message"
+
+
+def test_logging_async_nodes(caplog):
+    """Test logging of async nodes at at multiple logging levels."""
path: tests/plugins/test_logging.py
user: login: ellipsis-dev[bot]
body: Typo: The docstring in `test_logging_async_nodes` contains "at at multiple logging levels". Please remove the duplicate 'at' so that it reads "at multiple logging levels".
```suggestion
    """Test logging of async nodes at multiple logging levels."""
```
created_at: 2025-03-17T04:22:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,387 @@
+import asyncio
+import logging
+import sys
+
+import pytest
+
+from hamilton import ad_hoc_utils, async_driver, driver
+from hamilton.execution import executors
+from hamilton.htypes import Collect, Parallelizable
+from hamilton.plugins.h_dask import DaskExecutor
+from hamilton.plugins.h_logging import AsyncLoggingAdapter, LoggingAdapter, get_logger
+from hamilton.plugins.h_ray import RayTaskExecutor
+from hamilton.plugins.h_threadpool import FutureAdapter
+
+
+def _split_log_messages(caplog, name):
+    debug, info, warning, error = [], [], [], []
+    for record in caplog.records:
+        if record.name == name:
+            if record.levelno == logging.DEBUG:
+                debug.append(record.message)
+            elif record.levelno == logging.INFO:
+                info.append(record.message)
+            elif record.levelno == logging.WARNING:
+                warning.append(record.message)
+            elif record.levelno == logging.ERROR:
+                error.append(record.message)
+    return debug, info, warning, error
+
+
+def test_logging_serial_nodes_at_info_level(caplog):
+    """Test logging of serial nodes at INFO level - log order matters for this test."""
+
+    name = "test_logging_serial_nodes_at_info_level"
+    caplog.set_level(logging.INFO, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b(a: str) -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return a + " b"
+
+    def c(b: str) -> str:
+        return b + " c"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+    result = dr.execute(["c"])
+    assert result["c"] == "a b c"
+
+    messages = [record.message for record in caplog.records if record.name == name]
+
+    assert messages[0].startswith("Graph run")
+    assert messages[1:-1] == [
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Context aware message",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+    ]
+    assert messages[-1].endswith("- Finished graph execution [OK]")
+
+    levels = [record.levelname for record in caplog.records if record.name == name]
+    assert levels == ["INFO", "INFO", "WARNING", "INFO", "INFO", "INFO"]
+
+
+def test_logging_serial_nodes_at_debug_level(caplog):
+    """Test logging of serial nodes at DEBUG level - log order matters for this test."""
+
+    name = "test_logging_serial_nodes_at_debug_level"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b(a: str) -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return a + " b"
+
+    def c(b: str) -> str:
+        return b + " c"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c)
+    dr = driver.Builder().with_modules(modules).with_adapters(LoggingAdapter(name)).build()
+
+    result = dr.execute(["c"])
+    assert result["c"] == "a b c"
+
+    messages = [record.message for record in caplog.records if record.name == name]
+    assert messages[0].startswith("Graph run")
+    assert messages[1:-1] == [
+        "Node 'a' - Starting execution without dependencies",
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Starting execution with dependencies 'a'",
+        "Node 'b' - Context aware message",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Starting execution with dependencies 'b'",
+        "Node 'c' - Finished execution [OK]",
+    ]
+    assert messages[-1].endswith("- Finished graph execution [OK]")
+
+    levels = [record.levelname for record in caplog.records if record.name == name]
+    assert levels == ["INFO", "DEBUG", "INFO", "DEBUG", "WARNING", "INFO", "DEBUG", "INFO", "INFO"]
+
+
+@pytest.mark.parametrize("adapter", [None, FutureAdapter])
+def test_logging_branching_nodes(caplog, adapter):
+    """Test logging of branching nodes at multiple logging levels."""
+
+    name = "test_logging_branching_nodes"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def a() -> str:
+        return "a"
+
+    def b() -> str:
+        return "b"
+
+    def c() -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return "c"
+
+    def d(a: str, b: str) -> str:
+        return a + " " + b + " d"
+
+    def e(c: str) -> str:
+        return c + " e"
+
+    def f(d: str, e: str) -> str:
+        return d + " " + e + " f"
+
+    modules = ad_hoc_utils.create_temporary_module(a, b, c, d, e, f)
+    adapters = [LoggingAdapter(name)]
+    if adapter:
+        adapters.append(adapter())
+    dr = driver.Builder().with_modules(modules).with_adapters(*adapters).build()
+
+    result = dr.execute(["f"])
+    assert result["f"] == "a b d c e f"
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert set(info[1:-1]) == {
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+        "Node 'd' - Finished execution [OK]",
+        "Node 'e' - Finished execution [OK]",
+        "Node 'f' - Finished execution [OK]",
+    }
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(debug) == {
+        "Node 'a' - Starting execution without dependencies",
+        "Node 'b' - Starting execution without dependencies",
+        "Node 'c' - Starting execution without dependencies",
+        "Node 'd' - Starting execution with dependencies 'a', 'b'",
+        "Node 'e' - Starting execution with dependencies 'c'",
+        "Node 'f' - Starting execution with dependencies 'd', 'e'",
+    }
+
+    assert len(warning) == 1
+    assert warning[0] == "Node 'c' - Context aware message"
+
+
+def test_logging_async_nodes(caplog):
+    """Test logging of async nodes at at multiple logging levels."""
+
+    name = "test_logging_async_nodes"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    async def a() -> str:
+        return "a"
+
+    async def b() -> str:
+        return "b"
+
+    async def c() -> str:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return "c"
+
+    async def d(a: str, b: str) -> str:
+        return a + " " + b + " d"
+
+    async def e(c: str) -> str:
+        return c + " e"
+
+    async def f(d: str, e: str) -> str:
+        return d + " " + e + " f"
+
+    async def run_async(module, name):
+        dr = (
+            await async_driver.Builder()  # type: ignore
+            .with_modules(module)
+            .with_adapters(AsyncLoggingAdapter(name))
+            .build()
+        )
+        result = await dr.execute(["f"])
+        return result
+
+    module = ad_hoc_utils.create_temporary_module(a, b, c, d, e, f)
+    result = asyncio.run(run_async(module, name))
+
+    assert result["f"] == "a b d c e f"
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert set(info[1:-1]) == {
+        "Node 'a' - Finished execution [OK]",
+        "Node 'b' - Finished execution [OK]",
+        "Node 'c' - Finished execution [OK]",
+        "Node 'd' - Finished execution [OK]",
+        "Node 'e' - Finished execution [OK]",
+        "Node 'f' - Finished execution [OK]",
+    }
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(debug) == {
+        "Node 'a' - Submitting async node without dependencies",
+        "Node 'b' - Submitting async node without dependencies",
+        "Node 'c' - Submitting async node without dependencies",
+        "Node 'd' - Submitting async node with dependencies 'a', 'b'",
+        "Node 'e' - Submitting async node with dependencies 'c'",
+        "Node 'f' - Submitting async node with dependencies 'd', 'e'",
+    }
+
+    assert len(warning) == 1
+    assert warning[0] == "Node 'c' - Context aware message"
+
+
+@pytest.mark.parametrize(
+    ["executor_type", "executor_args"],
+    [
+        (executors.SynchronousLocalTaskExecutor, {}),
+        pytest.param(
+            executors.MultiProcessingExecutor,
+            {"max_tasks": 5},
+            marks=pytest.mark.skipif(
+                sys.platform == "win32", reason="Windows does not support fork"
+            ),
+        ),
+        (executors.MultiThreadingExecutor, {"max_tasks": 5}),
+        (RayTaskExecutor, {}),
+        (DaskExecutor, {"client": None}),
+    ],
+)
+def test_logging_parallel_nodes(caplog, executor_type, executor_args):
+    """Test logging of parallel nodes at multiple logging levels."""
+
+    # NOTE: These test is brittle, as it depends on undocumented names of the expanded tasks.
+
+    name = "test_logging_parallel_nodes_at_info_level"
+    caplog.set_level(logging.DEBUG, logger=name)
+
+    def b() -> int:
+        return 5
+
+    def c(b: int) -> Parallelizable[int]:
+        for i in range(b):
+            yield i
+
+    def d(c: int) -> int:
+        logger = get_logger(name)
+        logger.warning("Context aware message")
+        return 2 * c
+
+    def e(d: Collect[int]) -> int:
+        return sum(d)
+
+    def f(e: int) -> int:
+        return e
+
+    if executor_type == DaskExecutor:
+        import dask.distributed
+
+        cluster = dask.distributed.LocalCluster(n_workers=5)
+        client = dask.distributed.Client(cluster)
+        executor_args["client"] = client
+
+    modules = ad_hoc_utils.create_temporary_module(b, c, d, e, f)
+    adapters = [LoggingAdapter(name)]
+    dr = (
+        driver.Builder()
+        .with_modules(modules)
+        .with_adapters(*adapters)
+        .enable_dynamic_execution(allow_experimental_mode=True)
+        .with_remote_executor(executor_type(**executor_args))
+        .build()
+    )
+
+    result = dr.execute(["f"])
+    assert result["f"] == 20
+
+    debug, info, warning, _ = _split_log_messages(caplog, name)
+
+    assert info[0].startswith("Graph run")
+    assert info[1].endswith("task-based logging is enabled")
+    assert info[-1].endswith("- Finished graph execution [OK]")
+
+    assert set(info[2:-1]) == {
+        "Task 'b' - Task completed [OK]",
+        "Task 'expand-c' - Task completed [OK]",
+        "Task 'expand-c.0.block-c' - Task completed [OK]",
+        "Task 'expand-c.1.block-c' - Task completed [OK]",
+        "Task 'expand-c.2.block-c' - Task completed [OK]",
+        "Task 'expand-c.3.block-c' - Task completed [OK]",
+        "Task 'expand-c.4.block-c' - Task completed [OK]",
+        "Task 'collect-c' - Task completed [OK]",
+        "Task 'f' - Task completed [OK]",
+    }
+
+    # Note: Certain executors do not log task and node level debug messages (especially if they
+    # are not running in the same process as the driver).
+    local_debug_log = {
+        "Task 'b' - Initializing new task and submitting to executor",
+        "Task 'b' - Starting execution",
+        "Task 'b' - Starting execution without dependencies",
+        "Task 'b' - Node 'b' - Finished execution [OK]",
+        "Task 'b' - Finished execution [Ok]",
path: tests/plugins/test_logging.py
user: login: ellipsis-dev[bot]
body: Typographical inconsistency: Several log messages use `[Ok]` instead of the consistent `[OK]`. For instance, on line 325 (and similar lines 330, 340, 345), please update `[Ok]` to `[OK]` to match the rest of the code.
```suggestion
        "Task 'b' - Finished execution [OK]",
```
created_at: 2025-03-17T04:22:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,444 @@
+"""Synchronous/asynchronous adapter and functions for context-aware logging with Hamilton."""
path: hamilton/plugins/h_logging.py
user: login: elijahbenizzy
body: This could actually live in default - I think it uses only standard imports (and hamilton imports). Thoughts?
created_at: 2025-03-20T03:24:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,444 @@
+"""Synchronous/asynchronous adapter and functions for context-aware logging with Hamilton."""
path: hamilton/plugins/h_logging.py
user: login: cswartzvi
body: I originally was heading down that path, however there was one nagging issue that  made me change course - the `get_logger` function. It felt a little unnatural to import it from something like `hamilton.lifecycle`, especially when it's not very useful without the `LoggingAdapter` / `AsyncLoggingAdapter`. I could totally be overthinking this 🙃 
created_at: 2025-03-20T23:54:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,444 @@
+"""Synchronous/asynchronous adapter and functions for context-aware logging with Hamilton."""
path: hamilton/plugins/h_logging.py
user: login: elijahbenizzy
body: Ahh, yeah, I get that. I think it's good as a plugin, definitely not overthinking!

created_at: 2025-03-21T03:57:30Z
author_association: CONTRIBUTOR

PR comments:
user: login: cswartzvi
created_at: 2025-03-17T04:36:01Z
body: Ahh I see, `ray` and `dask` have separate testing environments  - I will disable those tests for now.
author_association: CONTRIBUTOR
user: login: cswartzvi
created_at: 2025-03-17T13:59:47Z
body: Tests are still failing due to `xgboost` (also in #1287) and an issue with my multiprocessing test (I had a feeling that one might cause problems, so much so I skipif-it on windows).
author_association: CONTRIBUTOR
user: login: cswartzvi
created_at: 2025-03-22T15:29:43Z
body: @elijahbenizzy I rebased on my branch from #1287 and updated the hook names
author_association: CONTRIBUTOR