Repository: esphome/esphome (2026-04-19T07:25:37Z)

Issue:
Regression in 2026.4.0: substitutions: !include file.yaml no longer works (worked in all previous versions)
### The problem

**Which version of ESPHome has the issue?**
2026.4.0

**Last working version:**
All versions before 2026.4.0

**What type of installation are you using?**
Manual / pip (ESPHome Studio by ATNEL)

**Describe the bug:**
The substitution system redesign in 2026.4.0 (PR #14918) introduced a regression.
The following syntax, which worked correctly in all previous versions, now fails:

substitutions: !include components/substitutions.yaml

Error message:
  Substitutions must be a key to value mapping, got <class 'esphome.helpers.IncludeFile'>

**Use case / Why this matters:**
This is a legitimate and widely used pattern for workspace-level shared substitution files.
Example architecture:

  workspace/
    EHSFONTS/
      mdi_glyph_substitutions.yaml   <-- shared across ALL projects (~7000 lines of MDI glyph definitions)
    project-A/
      project-A.yaml                 <-- substitutions: !include components/substitutions.yaml
      components/
        substitutions.yaml           <-- <<: !include ../../EHSFONTS/mdi_glyph_substitutions.yaml
    project-B/
      project-B.yaml
      components/
        substitutions.yaml

This pattern allows:
1. One shared glyph file for all projects (no duplication)
2. Per-project user substitutions in components/substitutions.yaml
3. Main YAML managed by a wizard/generator tool (user does not edit it directly)

Breaking this pattern forces either:
- Duplicating 7000 lines of glyph definitions into every single project
- Inlining substitutions into the wizard-managed main YAML (breaks the tool architecture)

**Minimal reproduction:**

File: mydevice.yaml
  substitutions: !include subs.yaml

File: subs.yaml
  my_variable: "hello"

**Expected behavior:**
Works as before - substitutions file is loaded and merged as a key-value map.

**Actual behavior:**
  Failed config
  substitutions: None
    {}
  Substitutions must be a key to value mapping, got <class 'esphome.helpers.IncludeFile'>

### Which version of ESPHome has the issue?

2026.4.0

### What type of installation are you using?

pip

### What platform are you using?

ESP8266

### Component causing the issue

substitutions

### YAML Config

```yaml
# Minimal reproduction - main file: mydevice.yaml
substitutions: !include components/substitutions.yaml

esphome:
  name: test-device

esp8266:
  board: d1_mini

# File: components/substitutions.yaml
<<: !include ../../EHSFONTS/mdi_glyph_substitutions.yaml


# File: EHSFONTS/mdi_glyph_substitutions.yaml (fragment)
mdi_ab_testing: "\U000F01C9"
mdi_abacus: "\U000F16E0"
# ... (thousands of MDI glyph definitions)
```

### Anything in the logs that might be useful for us?

```txt
INFO ESPHome 2026.4.0
INFO Reading configuration mydevice.yaml...
Failed config
substitutions: None
  {}
Substitutions must be a key to value mapping, got <class 'esphome.helpers.IncludeFile'>
```

### Additional information

This syntax worked correctly in all ESPHome versions before 2026.4.0.
The regression was introduced by the substitution system redesign in PR #14918.

Workspace structure used:
  workspace/
    EHSFONTS/
      mdi_glyph_substitutions.yaml   <-- ~7000 lines, shared across all projects
    project-A/
      project-A.yaml                 <-- substitutions: !include components/substitutions.yaml
      components/
        substitutions.yaml           <-- <<: !include ../../EHSFONTS/mdi_glyph_substitutions.yaml

This architecture allows one shared glyph file for all projects without duplication.
Breaking this forces copying 7000 lines into every single project - unacceptable.


Discussion:

Patch:
diff --git a/esphome/components/packages/__init__.py b/esphome/components/packages/__init__.py
index 3f3df753512c..252a24061a08 100644
--- a/esphome/components/packages/__init__.py
+++ b/esphome/components/packages/__init__.py
@@ -10,6 +10,7 @@
     ContextVars,
     push_context,
     resolve_include,
+    resolve_substitutions_block,
     substitute,
 )
 from esphome.components.substitutions.jinja import has_jinja
@@ -516,7 +517,12 @@ def do_packages_pass(
     if CONF_PACKAGES not in config:
         return config
 
-    substitutions = UserDict(config.pop(CONF_SUBSTITUTIONS, {}))
+    with cv.prepend_path(CONF_SUBSTITUTIONS):
+        substitutions = UserDict(
+            resolve_substitutions_block(
+                config.pop(CONF_SUBSTITUTIONS, {}), command_line_substitutions
+            )
+        )
     processor = _PackageProcessor(
         substitutions, command_line_substitutions, skip_update
     )
diff --git a/esphome/components/substitutions/__init__.py b/esphome/components/substitutions/__init__.py
index 94aebbbfe356..e451ad5db8d3 100644
--- a/esphome/components/substitutions/__init__.py
+++ b/esphome/components/substitutions/__init__.py
@@ -414,6 +414,34 @@ def _warn_unresolved_variables(errors: ErrList) -> None:
         )
 
 
+def resolve_substitutions_block(
+    substitutions: Any,
+    command_line_substitutions: dict[str, Any] | None,
+) -> dict[str, Any]:
+    """Resolve a deferred ``substitutions: !include file.yaml`` and validate the shape.
+
+    The caller is responsible for wrapping the call in
+    ``cv.prepend_path(CONF_SUBSTITUTIONS)`` for error reporting.
+    ``command_line_substitutions`` seeds the filename context so
+    ``substitutions: !include ${var}.yaml`` can reference CLI-provided vars.
+    """
+    if isinstance(substitutions, IncludeFile):
+        # Single-shot resolution — matches ``_walk_packages`` for the
+        # ``packages: !include`` entry point.  Chained includes (an include that
+        # itself loads another ``!include`` at the top level) are not supported.
+        substitutions, _ = resolve_include(
+            substitutions,
+            [],
+            ContextVars(command_line_substitutions or {}),
+            strict_undefined=False,
+        )
+    if not isinstance(substitutions, dict):
+        raise cv.Invalid(
+            f"Substitutions must be a key to value mapping, got {type(substitutions)}"
+        )
+    return substitutions
+
+
 def do_substitution_pass(
     config: OrderedDict, command_line_substitutions: dict[str, Any] | None = None
 ) -> OrderedDict:
@@ -429,10 +457,9 @@ def do_substitution_pass(
     # Use merge_dicts_ordered to preserve OrderedDict type for move_to_end()
     substitutions = config.pop(CONF_SUBSTITUTIONS, {})
     with cv.prepend_path(CONF_SUBSTITUTIONS):
-        if not isinstance(substitutions, dict):
-            raise cv.Invalid(
-                f"Substitutions must be a key to value mapping, got {type(substitutions)}"
-            )
+        substitutions = resolve_substitutions_block(
+            substitutions, command_line_substitutions
+        )
         substitutions = merge_dicts_ordered(
             substitutions, command_line_substitutions or {}
         )


Test patch:
diff --git a/tests/unit_tests/fixtures/substitutions/15-substitutions_as_include.approved.yaml b/tests/unit_tests/fixtures/substitutions/15-substitutions_as_include.approved.yaml
new file mode 100644
index 000000000000..14aa707def5c
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/15-substitutions_as_include.approved.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  wifi_password: sub_password
+wifi:
+  ssid: main_ssid
+  password: sub_password
diff --git a/tests/unit_tests/fixtures/substitutions/15-substitutions_as_include.input.yaml b/tests/unit_tests/fixtures/substitutions/15-substitutions_as_include.input.yaml
new file mode 100644
index 000000000000..5909e7bf4f7f
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/15-substitutions_as_include.input.yaml
@@ -0,0 +1,5 @@
+substitutions: !include 15-substitutions_inc.yaml
+
+wifi:
+  ssid: main_ssid
+  password: $wifi_password
diff --git a/tests/unit_tests/fixtures/substitutions/15-substitutions_inc.yaml b/tests/unit_tests/fixtures/substitutions/15-substitutions_inc.yaml
new file mode 100644
index 000000000000..44d9a4b9ef29
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/15-substitutions_inc.yaml
@@ -0,0 +1,1 @@
+wifi_password: sub_password
diff --git a/tests/unit_tests/fixtures/substitutions/16-substitutions_as_include_with_packages.approved.yaml b/tests/unit_tests/fixtures/substitutions/16-substitutions_as_include_with_packages.approved.yaml
new file mode 100644
index 000000000000..14aa707def5c
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/16-substitutions_as_include_with_packages.approved.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  wifi_password: sub_password
+wifi:
+  ssid: main_ssid
+  password: sub_password
diff --git a/tests/unit_tests/fixtures/substitutions/16-substitutions_as_include_with_packages.input.yaml b/tests/unit_tests/fixtures/substitutions/16-substitutions_as_include_with_packages.input.yaml
new file mode 100644
index 000000000000..a2e72f33a232
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/16-substitutions_as_include_with_packages.input.yaml
@@ -0,0 +1,9 @@
+substitutions: !include 15-substitutions_inc.yaml
+
+packages:
+  wifi_pkg:
+    wifi:
+      password: $wifi_password
+
+wifi:
+  ssid: main_ssid
diff --git a/tests/unit_tests/fixtures/substitutions/17-substitutions_include_cli_var.approved.yaml b/tests/unit_tests/fixtures/substitutions/17-substitutions_include_cli_var.approved.yaml
new file mode 100644
index 000000000000..f1fd5fb07862
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/17-substitutions_include_cli_var.approved.yaml
@@ -0,0 +1,6 @@
+substitutions:
+  subs_file: 15-substitutions_inc
+  wifi_password: sub_password
+wifi:
+  ssid: main_ssid
+  password: sub_password
diff --git a/tests/unit_tests/fixtures/substitutions/17-substitutions_include_cli_var.input.yaml b/tests/unit_tests/fixtures/substitutions/17-substitutions_include_cli_var.input.yaml
new file mode 100644
index 000000000000..3248504b4681
--- /dev/null
+++ b/tests/unit_tests/fixtures/substitutions/17-substitutions_include_cli_var.input.yaml
@@ -0,0 +1,8 @@
+command_line_substitutions:
+  subs_file: 15-substitutions_inc
+
+substitutions: !include ${subs_file}.yaml
+
+wifi:
+  ssid: main_ssid
+  password: $wifi_password
diff --git a/tests/unit_tests/test_substitutions.py b/tests/unit_tests/test_substitutions.py
index 01c669e54204..71bbd9db86df 100644
--- a/tests/unit_tests/test_substitutions.py
+++ b/tests/unit_tests/test_substitutions.py
@@ -675,6 +675,57 @@ def test_include_filename_substitution_undefined_var(tmp_path: Path) -> None:
         substitutions.do_substitution_pass(config)
 
 
+def test_do_substitution_pass_included_substitutions_must_be_mapping(
+    tmp_path: Path,
+) -> None:
+    """`substitutions: !include list.yaml` where the file holds a list raises cv.Invalid.
+
+    Locks in the shape check that runs after the deferred IncludeFile has been
+    resolved.
+    """
+    parent = tmp_path / "main.yaml"
+    parent.write_text("")
+
+    def loader(path: Path):
+        return ["not", "a", "mapping"]
+
+    include = yaml_util.IncludeFile(parent, "subs.yaml", None, loader)
+    config = OrderedDict({CONF_SUBSTITUTIONS: include})
+
+    with pytest.raises(
+        cv.Invalid, match="Substitutions must be a key to value mapping"
+    ):
+        substitutions.do_substitution_pass(config)
+
+
+def test_do_packages_pass_included_substitutions_must_be_mapping(
+    tmp_path: Path,
+) -> None:
+    """`substitutions: !include list.yaml` alongside `packages:` raises cv.Invalid.
+
+    Without the shape check, ``UserDict(...)`` would surface a low-level
+    ``TypeError``; the explicit ``cv.Invalid`` points at the substitutions path.
+    """
+    parent = tmp_path / "main.yaml"
+    parent.write_text("")
+
+    def loader(path: Path):
+        return ["not", "a", "mapping"]
+
+    include = yaml_util.IncludeFile(parent, "subs.yaml", None, loader)
+    config = OrderedDict(
+        {
+            CONF_SUBSTITUTIONS: include,
+            "packages": {"noop": {"wifi": {"ssid": "main"}}},
+        }
+    )
+
+    with pytest.raises(
+        cv.Invalid, match="Substitutions must be a key to value mapping"
+    ):
+        do_packages_pass(config)
+
+
 def test_resolve_package_undefined_var_in_include_filename(tmp_path: Path) -> None:
     """An undefined substitution in a package include filename raises cv.Invalid.