Repository: copier-org/copier
Pull request #1855 (2024-11-10 03:24:14)

feat: add dynamic file structures in loop using yield-tag
What I did:
- Added a new Jinja extension in `copier/jinja_ext.py` to handle `yield` tags
  - This is a lightweight extension that detects `yield` tags and sets up the necessary context through `jinja_env`
- Updated `_render_path` to return multiple destination paths and their corresponding contexts when `yield` tags are present
  - Internally, delegated the recursive logic for generating paths and contexts from `yield` tags to the `_render_parts` function
- Added `extra_context` parameter to `_render_file` to enable injection of contexts generated from `yield` tags

kj-9: hopefully closes #1271
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/copier-org/copier/pull/1855?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 93.92%. Comparing base [(`8391966`)](https://app.codecov.io/gh/copier-org/copier/commit/83919666a2d7000d7d4f6b0ef8cd68358dfd198c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) to head [(`7164aa7`)](https://app.codecov.io/gh/copier-org/copier/commit/7164aa76270113f7246b430ad959cbe789a95019?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org).
> Report is 2 commits behind head on master.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1855      +/-   ##
==========================================
- Coverage   97.67%   93.92%   -3.75%     
==========================================
  Files          49       51       +2     
  Lines        5238     5349     +111     
==========================================
- Hits         5116     5024      -92     
- Misses        122      325     +203     
```

| [Flag](https://app.codecov.io/gh/copier-org/copier/pull/1855/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) | Coverage Δ | |
|---|---|---|
| [unittests](https://app.codecov.io/gh/copier-org/copier/pull/1855/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) | `93.92% <100.00%> (-3.75%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/copier-org/copier/pull/1855?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org).

yajo: I only reviewed tests in depth so far, and it seems the interface is almost exactly what I meant! Awesome! 

Quoting from https://github.com/copier-org/copier/issues/1271#issue-1821616806:

> are you rendering a filename?
> No: just let the exception raise and print a nice message "Using yield outside of file names isn't supported".

That seems to be the only missing part. At least I saw no test for that. Could you add this please? 

When done I'll review the implementation. This PR is big!

Thank you very much 😊
kj-9: @yajo 

Thank you for your review!

I've added a new commit that implements the behavior you mentioned - throwing an exception when yield is used for rendering file contents. I've also added the corresponding test case.

Looking forward to your implementation review!
kj-9: @yajo 
Thank you for the quick review!

I have added a few commits:
- Fixes according to your comments
- Minor code refactoring

And in the last commit, I added documentation. Specifically, three points:

1. Added a row for the new feature in the comparison table (as far as I know, only copier supports this feature)
2. Added `jinja_ext.py` to the `Reference`
3. Added a section explaining the new feature to `Configuring`.

This is my first time adding documentation, so I'm not sure if it meets your expectations.
Could you please review it again?
sisp: I'll try to give feedback later today.

Patch:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index a03f2eb..abb833b 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -127,6 +127,12 @@ Ready to contribute? Here's how to set up the project for local development.
     pipx run pyclean . --debris --verbose
     ```
 
+    or
+
+    ```shell
+    uvx pyclean . --debris --verbose
+    ```
+
 1.  Commit your changes and push your branch to GitHub:
 
     ```shell
diff --git a/README.md b/README.md
index 9886b09..f46c1cf 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,8 @@ A library and CLI app for rendering project templates.
 
 1. Install Python 3.9 or newer.
 1. Install Git 2.27 or newer.
-1. To use as a CLI app: `pipx install copier`
+1. To use as a CLI app: [`pipx install copier`](https://github.com/pypa/pipx) or
+   [`uv tool install copier`](https://docs.astral.sh/uv/#tool-management)
 1. To use as a library: `pip install copier` or `conda install -c conda-forge copier`
 
 ### Nix flake
diff --git a/copier/errors.py b/copier/errors.py
index 05d4180..d74bd41 100644
--- a/copier/errors.py
+++ b/copier/errors.py
@@ -112,6 +112,14 @@ class UnsafeTemplateError(CopierError):
         )
 
 
+class YieldTagInFileError(CopierError):
+    """A yield tag is used in the file content, but it is not allowed."""
+
+
+class MultipleYieldTagsError(CopierError):
+    """Multiple yield tags are used in one path name, but it is not allowed."""
+
+
 # Warnings
 class CopierWarning(Warning):
     """Base class for all other Copier warnings."""
diff --git a/copier/jinja_ext.py b/copier/jinja_ext.py
new file mode 100644
index 0000000..24d8f83
--- /dev/null
+++ b/copier/jinja_ext.py
@@ -0,0 +1,123 @@
+"""Jinja2 extensions built for Copier."""
+
+from __future__ import annotations
+
+from typing import Any, Callable, Iterable
+
+from jinja2 import nodes
+from jinja2.exceptions import UndefinedError
+from jinja2.ext import Extension
+from jinja2.parser import Parser
+from jinja2.sandbox import SandboxedEnvironment
+
+from copier.errors import MultipleYieldTagsError
+
+
+class YieldEnvironment(SandboxedEnvironment):
+    """Jinja2 environment with attributes from the YieldExtension.
+
+    This is simple environment class that extends the SandboxedEnvironment
+    for use with the YieldExtension, mainly for avoiding type errors.
+
+    We use the SandboxedEnvironment because we want to minimize the risk of hidden malware
+    in the templates. Of course we still have the post-copy tasks to worry about, but at least
+    they are more visible to the final user.
+    """
+
+    yield_name: str | None
+    yield_iterable: Iterable[Any] | None
+
+    def __init__(self, *args: Any, **kwargs: Any):
+        super().__init__(*args, **kwargs)
+        self.extend(yield_name=None, yield_iterable=None)
+
+
+class YieldExtension(Extension):
+    """Jinja2 extension for the `yield` tag.
+
+    If `yield` tag is used in a template, this extension sets following attribute to the
+    jinja environment:
+
+    - `yield_name`: The name of the variable that will be yielded.
+    - `yield_iterable`: The variable that will be looped over.
+
+    Note that this extension just sets the attributes but renders templates as usual.
+    It is the caller's responsibility to use the `yield_context` attribute in the template to
+    generate the desired output.
+
+    !!! example
+
+        ```pycon
+        >>> from copier.jinja_ext import YieldEnvironment, YieldExtension
+        >>> env = YieldEnvironment(extensions=[YieldExtension])
+        >>> template = env.from_string("{% yield single_var from looped_var %}{{ single_var }}{% endyield %}")
+        >>> template.render({"looped_var": [1, 2, 3]})
+        ''
+        >>> env.yield_name
+        'single_var'
+        >>> env.yield_iterable
+        [1, 2, 3]
+        ```
+    """
+
+    tags = {"yield"}
+
+    environment: YieldEnvironment
+
+    def preprocess(
+        self, source: str, _name: str | None, _filename: str | None = None
+    ) -> str:
+        """Preprocess hook to reset attributes before rendering."""
+        self.environment.yield_name = self.environment.yield_iterable = None
+
+        return source
+
+    def parse(self, parser: Parser) -> nodes.Node:
+        """Parse the `yield` tag."""
+        lineno = next(parser.stream).lineno
+
+        yield_name: nodes.Name = parser.parse_assign_target(name_only=True)
+        parser.stream.expect("name:from")
+        yield_iterable = parser.parse_expression()
+        body = parser.parse_statements(("name:endyield",), drop_needle=True)
+
+        return nodes.CallBlock(
+            self.call_method(
+                "_yield_support",
+                [nodes.Const(yield_name.name), yield_iterable],
+            ),
+            [],
+            [],
+            body,
+            lineno=lineno,
+        )
+
+    def _yield_support(
+        self, yield_name: str, yield_iterable: Iterable[Any], caller: Callable[[], str]
+    ) -> str:
+        """Support function for the yield tag.
+
+        Sets the `yield_name` and `yield_iterable` attributes in the environment then calls
+        the provided caller function. If an UndefinedError is raised, it returns an empty string.
+        """
+        if (
+            self.environment.yield_name is not None
+            or self.environment.yield_iterable is not None
+        ):
+            raise MultipleYieldTagsError(
+                "Attempted to parse the yield tag twice. Only one yield tag is allowed per path name.\n"
+                f'A yield tag with the name: "{self.environment.yield_name}" and iterable: "{self.environment.yield_iterable}" already exists.'
+            )
+
+        self.environment.yield_name = yield_name
+        self.environment.yield_iterable = yield_iterable
+
+        try:
+            res = caller()
+
+        # expression like `dict.attr` will always raise UndefinedError
+        # so we catch it here and return an empty string
+        except UndefinedError:
+            res = ""
+
+        return res
diff --git a/copier/main.py b/copier/main.py
index 351e42f..f2b1db1 100644
--- a/copier/main.py
+++ b/copier/main.py
@@ -29,7 +29,6 @@ from typing import (
 from unicodedata import normalize
 
 from jinja2.loaders import FileSystemLoader
-from jinja2.sandbox import SandboxedEnvironment
 from pathspec import PathSpec
 from plumbum import ProcessExecutionError, colors
 from plumbum.cli.terminal import ask
@@ -44,7 +43,9 @@ from .errors import (
     ExtensionNotFoundError,
     UnsafeTemplateError,
     UserMessageError,
+    YieldTagInFileError,
 )
+from .jinja_ext import YieldEnvironment, YieldExtension
 from .subproject import Subproject
 from .template import Task, Template
 from .tools import (
@@ -541,7 +542,7 @@ class Worker:
         return self.template.exclude + tuple(self.exclude)
 
     @cached_property
-    def jinja_env(self) -> SandboxedEnvironment:
+    def jinja_env(self) -> YieldEnvironment:
         """Return a pre-configured Jinja environment.
 
         Respects template settings.
@@ -550,14 +551,11 @@ class Worker:
         loader = FileSystemLoader(paths)
         default_extensions = [
             "jinja2_ansible_filters.AnsibleCoreFiltersExtension",
+            YieldExtension,
         ]
         extensions = default_extensions + list(self.template.jinja_extensions)
-        # We want to minimize the risk of hidden malware in the templates
-        # so we use the SandboxedEnvironment instead of the regular one.
-        # Of course we still have the post-copy tasks to worry about, but at least
-        # they are more visible to the final user.
         try:
-            env = SandboxedEnvironment(
+            env = YieldEnvironment(
                 loader=loader, extensions=extensions, **self.template.envops
             )
         except ModuleNotFoundError as error:
@@ -607,19 +605,25 @@ class Worker:
         for src in scantree(str(self.template_copy_root), follow_symlinks):
             src_abspath = Path(src.path)
             src_relpath = Path(src_abspath).relative_to(self.template.local_abspath)
-            dst_relpath = self._render_path(
+            dst_relpaths_ctxs = self._render_path(
                 Path(src_abspath).relative_to(self.template_copy_root)
             )
-            if dst_relpath is None or self.match_exclude(dst_relpath):
-                continue
-            if src.is_symlink() and self.template.preserve_symlinks:
-                self._render_symlink(src_relpath, dst_relpath)
-            elif src.is_dir(follow_symlinks=follow_symlinks):
-                self._render_folder(dst_relpath)
-            else:
-                self._render_file(src_relpath, dst_relpath)
+            for dst_relpath, ctx in dst_relpaths_ctxs:
+                if self.match_exclude(dst_relpath):
+                    continue
+                if src.is_symlink() and self.template.preserve_symlinks:
+                    self._render_symlink(src_relpath, dst_relpath)
+                elif src.is_dir(follow_symlinks=follow_symlinks):
+                    self._render_folder(dst_relpath)
+                else:
+                    self._render_file(src_relpath, dst_relpath, extra_context=ctx or {})
 
-    def _render_file(self, src_relpath: Path, dst_relpath: Path) -> None:
+    def _render_file(
+        self,
+        src_relpath: Path,
+        dst_relpath: Path,
+        extra_context: AnyByStrDict | None = None,
+    ) -> None:
         """Render one file.
 
         Args:
@@ -629,6 +633,8 @@ class Worker:
             dst_relpath:
                 File to be created. It must be a path relative to the subproject
                 root.
+            extra_context:
+                Additional variables to use for rendering the template.
         """
         # TODO Get from main.render_file()
         assert not src_relpath.is_absolute()
@@ -644,7 +650,13 @@ class Worker:
                 # suffix is empty, fallback to copy
                 new_content = src_abspath.read_bytes()
             else:
-                new_content = tpl.render(**self._render_context()).encode()
+                new_content = tpl.render(
+                    **self._render_context(), **(extra_context or {})
+                ).encode()
+                if self.jinja_env.yield_name:
+                    raise YieldTagInFileError(
+                        f"File {src_relpath} contains a yield tag, but it is not allowed."
+                    )
         else:
             new_content = src_abspath.read_bytes()
         dst_abspath = self.subproject.local_abspath / dst_relpath
@@ -716,8 +728,85 @@ class Worker:
             dst_abspath = self.subproject.local_abspath / dst_relpath
             dst_abspath.mkdir(parents=True, exist_ok=True)
 
-    def _render_path(self, relpath: Path) -> Path | None:
-        """Render one relative path.
+    def _adjust_rendered_part(self, rendered_part: str) -> str:
+        """Adjust the rendered part if necessary.
+
+        If `{{ _copier_conf.answers_file }}` becomes the full path,
+        restore part to be just the end leaf.
+
+        Args:
+            rendered_part:
+                The rendered part of the path to adjust.
+
+        """
+        if str(self.answers_relpath) == rendered_part:
+            return Path(rendered_part).name
+        return rendered_part
+
+    def _render_parts(
+        self,
+        parts: tuple[str, ...],
+        rendered_parts: tuple[str, ...] | None = None,
+        extra_context: AnyByStrDict | None = None,
+        is_template: bool = False,
+    ) -> Iterable[tuple[Path, AnyByStrDict | None]]:
+        """Render a set of parts into path and context pairs.
+
+        If a yield tag is found in a part, it will recursively yield multiple path and context pairs.
+        """
+        if rendered_parts is None:
+            rendered_parts = tuple()
+
+        if not parts:
+            rendered_path = Path(*rendered_parts)
+
+            templated_sibling = (
+                self.template.local_abspath
+                / f"{rendered_path}{self.template.templates_suffix}"
+            )
+            if is_template or not templated_sibling.exists():
+                yield rendered_path, extra_context
+
+            return
+
+        part = parts[0]
+        parts = parts[1:]
+
+        if not extra_context:
+            extra_context = {}
+
+        # If the `part` has a yield tag, `self.jinja_env` will be set with the yield name and iterable
+        rendered_part = self._render_string(part, extra_context=extra_context)
+
+        yield_name = self.jinja_env.yield_name
+        if yield_name:
+            for value in self.jinja_env.yield_iterable or ():
+                new_context = {**extra_context, yield_name: value}
+                rendered_part = self._render_string(part, extra_context=new_context)
+                rendered_part = self._adjust_rendered_part(rendered_part)
+
+                # Skip if any part is rendered as an empty string
+                if not rendered_part:
+                    continue
+
+                yield from self._render_parts(
+                    parts, rendered_parts + (rendered_part,), new_context, is_template
+                )
+
+            return
+
+        # Skip if any part is rendered as an empty string
+        if not rendered_part:
+            return
+
+        rendered_part = self._adjust_rendered_part(rendered_part)
+
+        yield from self._render_parts(
+            parts, rendered_parts + (rendered_part,), extra_context, is_template
+        )
+
+    def _render_path(self, relpath: Path) -> Iterable[tuple[Path, AnyByStrDict | None]]:
+        """Render one relative path into multiple path and context pairs.
 
         Args:
             relpath:
@@ -729,29 +818,11 @@ class Worker:
         )
         # With an empty suffix, the templated sibling always exists.
         if templated_sibling.exists() and self.template.templates_suffix:
-            return None
+            return
         if self.template.templates_suffix and is_template:
             relpath = relpath.with_suffix("")
-        rendered_parts = []
-        for part in relpath.parts:
-            # Skip folder if any part is rendered as an empty string
-            part = self._render_string(part)
-            if not part:
-                return None
-            # {{ _copier_conf.answers_file }} becomes the full path; in that case,
-            # restore part to be just the end leaf
-            if str(self.answers_relpath) == part:
-                part = Path(part).name
-            rendered_parts.append(part)
-        result = Path(*rendered_parts)
-        if not is_template:
-            templated_sibling = (
-                self.template.local_abspath
-                / f"{result}{self.template.templates_suffix}"
-            )
-            if templated_sibling.exists():
-                return None
-        return result
+
+        yield from self._render_parts(relpath.parts, is_template=is_template)
 
     def _render_string(
         self, string: str, extra_context: AnyByStrDict | None = None
@@ -1078,6 +1149,8 @@ class Worker:
                 (apply_cmd << diff)(retcode=None)
                 if self.conflict == "inline":
                     conflicted = []
+                    old_path = Path(old_copy)
+                    new_path = Path(new_copy)
                     status = git("status", "--porcelain").strip().splitlines()
                     for line in status:
                         # Filter merge rejections (part 1/2)
@@ -1104,8 +1177,8 @@ class Worker:
                             "-L",
                             "after updating",
                             fname,
-                            Path(old_copy) / fname,
-                            Path(new_copy) / fname,
+                            old_path / fname,
+                            new_path / fname,
                             retcode=None,
                         )
                         # Remove rejection witness
@@ -1120,11 +1193,13 @@ class Worker:
                                 for line in conflicts_candidate
                             ):
                                 conflicted.append(fname)
-                    # Forcefully mark files with conflict markers as unmerged,
-                    # see SO post: https://stackoverflow.com/questions/77391627/
+                    # We ran `git merge-file` outside of a regular merge operation,
+                    # which means no merge conflict is recorded in the index.
+                    # Only the usual stage 0 is recorded, with the hash of the current version.
+                    # We therefore update the index with the missing stages:
+                    # 1 = current (before updating), 2 = base (last update), 3 = other (after updating).
+                    # See this SO post: https://stackoverflow.com/questions/79309642/
                     # and Git docs: https://git-scm.com/docs/git-update-index#_using_index_info.
-                    # For each file with conflict markers, we update the index to add
-                    # higher order versions of their paths, without entries for resolved contents.
                     if conflicted:
                         input_lines = []
                         for line in (
@@ -1133,9 +1208,25 @@ class Worker:
                             perms_sha_mode, path = line.split("\t")
                             perms, sha, _ = perms_sha_mode.split()
                             input_lines.append(f"0 {'0' * 40}\t{path}")
-                            input_lines.extend(
-                                f"{perms} {sha} {mode}\t{path}" for mode in (1, 2, 3)
-                            )
+                            input_lines.append(f"{perms} {sha} 1\t{path}")
+                            with suppress(ProcessExecutionError):
+                                # The following command will fail
+                                # if the file did not exist in the previous version.
+                                old_sha = git(
+                                    "hash-object",
+                                    "-w",
+                                    old_path / normalize_git_path(path),
+                                ).strip()
+                                input_lines.append(f"{perms} {old_sha} 2\t{path}")
+                            with suppress(ProcessExecutionError):
+                                # The following command will fail
+                                # if the file was deleted in the latest version.
+                                new_sha = git(
+                                    "hash-object",
+                                    "-w",
+                                    new_path / normalize_git_path(path),
+                                ).strip()
+                                input_lines.append(f"{perms} {new_sha} 3\t{path}")
                         (
                             git["update-index", "--index-info"]
                             << "\n".join(input_lines)
diff --git a/docs/comparisons.md b/docs/comparisons.md
index bb9961c..cd8417c 100644
--- a/docs/comparisons.md
+++ b/docs/comparisons.md
@@ -18,6 +18,7 @@ docs! We don't want to be biased, but it's easy that we tend to be:
 | Feature                                  | Copier                           | Cookiecutter                    | Yeoman        |
 | ---------------------------------------- | -------------------------------- | ------------------------------- | ------------- |
 | Can template file names                  | Yes                              | Yes                             | Yes           |
+| Can generate file structures in loops    | Yes                              | No                              | No            |
 | Configuration                            | Single YAML file[^1]             | Single JSON file                | JS module     |
 | Migrations                               | Yes                              | No                              | No            |
 | Programmed in                            | Python                           | Python                          | NodeJS        |
diff --git a/docs/configuring.md b/docs/configuring.md
index 636f04e..2da6f47 100644
--- a/docs/configuring.md
+++ b/docs/configuring.md
@@ -1013,6 +1013,8 @@ on them, so they are always installed when Copier is installed.
 
     # if Copier was installed with pipx
     pipx inject copier jinja2-time
+    # if Copier was installed with uv
+    uv tool install --with jinja2-time copier
     ```
 
 !!! example
diff --git a/docs/creating.md b/docs/creating.md
index d0415f3..75c07a8 100644
--- a/docs/creating.md
+++ b/docs/creating.md
@@ -130,3 +130,75 @@ The name of the project root directory.
 Some rendering contexts provide variables unique to them:
 
 -   [`migrations`](configuring.md#migrations)
+
+## Loop over lists to generate files and directories
+
+You can use the special `yield` tag in file and directory names to generate multiple
+files or directories based on a list of items.
+
+In the path name, `{% yield item from list_of_items %}{{ item }}{% endyield %}` will
+loop over the `list_of_items` and replace `{{ item }}` with each item in the list.
+
+A looped `{{ item }}` will be available in the scope of generated files and directories.
+
+```yaml title="copier.yml"
+commands:
+    type: yaml
+    multiselect: true
+    choices:
+        init:
+            value: &init
+                name: init
+                subcommands:
+                    - config
+                    - database
+        run:
+            value: &run
+                name: run
+                subcommands:
+                    - server
+                    - worker
+        deploy:
+            value: &deploy
+                name: deploy
+                subcommands:
+                    - staging
+                    - production
+    default: [*init, *run, *deploy]
+```
+
+```tree result="shell"
+commands
+    {% yield cmd from commands %}{{ cmd.name }}{% endyield %}
+        __init__.py
+        {% yield subcmd from cmd.subcommands %}{{ subcmd }}{% endyield %}.py.jinja
+```
+
+```python+jinja title="{% yield subcmd from cmd.subcommands %}{{ subcmd }}{% endyield %}.py.jinja"
+print("This is the `{{ subcmd }}` subcommand in the `{{ cmd.name }}` command")
+```
+
+If you answer with the default to the question, Copier will generate the following
+structure:
+
+```tree result="shell"
+commands
+    deploy
+        __init__.py
+        production.py
+        staging.py
+    init
+        __init__.py
+        config.py
+        database.py
+    run
+        __init__.py
+        server.py
+        worker.py
+```
+
+Where looped variables `cmd` and `subcmd` are rendered in generated files:
+
+```python title="commands/init/config.py"
+print("This is the `config` subcommand in the `init` command")
+```
diff --git a/docs/reference/jinja_ext.md b/docs/reference/jinja_ext.md
new file mode 100644
index 0000000..e78411e
--- /dev/null
+++ b/docs/reference/jinja_ext.md
@@ -0,0 +1,1 @@
+::: copier.jinja_ext
diff --git a/mkdocs.yml b/mkdocs.yml
index 81d1cfd..4eeed47 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -14,6 +14,7 @@ nav:
   - Reference:
       - cli.py: "reference/cli.md"
       - errors.py: "reference/errors.md"
+      - jinja_ext.py: "reference/jinja_ext.md"
       - main.py: "reference/main.md"
       - subproject.py: "reference/subproject.md"
       - template.py: "reference/template.md"
diff --git a/poetry.lock b/poetry.lock
index 707893f..7deb3e6 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -667,13 +667,13 @@ pyyaml = ">=5.1"
 
 [[package]]
 name = "mkdocs-material"
-version = "9.5.49"
+version = "9.5.50"
 description = "Documentation that simply works"
 optional = false
 python-versions = ">=3.8"
 files = [
-    {file = "mkdocs_material-9.5.49-py3-none-any.whl", hash = "sha256:c3c2d8176b18198435d3a3e119011922f3e11424074645c24019c2dcf08a360e"},
-    {file = "mkdocs_material-9.5.49.tar.gz", hash = "sha256:3671bb282b4f53a1c72e08adbe04d2481a98f85fed392530051f80ff94a9621d"},
+    {file = "mkdocs_material-9.5.50-py3-none-any.whl", hash = "sha256:f24100f234741f4d423a9d672a909d859668a4f404796be3cf035f10d6050385"},
+    {file = "mkdocs_material-9.5.50.tar.gz", hash = "sha256:ae5fe16f3d7c9ccd05bb6916a7da7420cf99a9ce5e33debd9d40403a090d5825"},
 ]
 
 [package.dependencies]
@@ -690,7 +690,7 @@ regex = ">=2022.4"
 requests = ">=2.26,<3.0"
 
 [package.extras]
-git = ["mkdocs-git-committers-plugin-2 (>=1.1,<2.0)", "mkdocs-git-revision-date-localized-plugin (>=1.2.4,<2.0)"]
+git = ["mkdocs-git-committers-plugin-2 (>=1.1,<3)", "mkdocs-git-revision-date-localized-plugin (>=1.2.4,<2.0)"]
 imaging = ["cairosvg (>=2.6,<3.0)", "pillow (>=10.2,<11.0)"]
 recommended = ["mkdocs-minify-plugin (>=0.7,<1.0)", "mkdocs-redirects (>=1.2,<2.0)", "mkdocs-rss-plugin (>=1.6,<2.0)"]
 
@@ -946,13 +946,13 @@ test = ["coverage[toml]", "paramiko", "psutil", "pytest (>=6.0)", "pytest-cov",
 
 [[package]]
 name = "poethepoet"
-version = "0.32.0"
+version = "0.32.1"
 description = "A task runner that works well with poetry."
 optional = false
 python-versions = ">=3.9"
 files = [
-    {file = "poethepoet-0.32.0-py3-none-any.whl", hash = "sha256:fba84c72d923feac228d1ea7734c5a54701f2e71fad42845f027c0fbf998a073"},
-    {file = "poethepoet-0.32.0.tar.gz", hash = "sha256:a700be02e932e1a8907ae630928fc769ea9a77986189ba6867e6e3fd8f60e5b7"},
+    {file = "poethepoet-0.32.1-py3-none-any.whl", hash = "sha256:d1e0a52a2f677870fac17dfb26bfe4910242756ac821443ef31f90ad26227c2d"},
+    {file = "poethepoet-0.32.1.tar.gz", hash = "sha256:471e1a025812dcd3d2997e30989681be5ab0a49232ee5fba94859629671c9584"},
 ]
 
 [package.dependencies]
@@ -961,17 +961,17 @@ pyyaml = ">=6.0.2,<7.0.0"
 tomli = {version = ">=1.2.2", markers = "python_version < \"3.11\""}
 
 [package.extras]
-poetry-plugin = ["poetry (>=1.0,<2.0)"]
+poetry-plugin = ["poetry (>=1.0,<3.0)"]
 
 [[package]]
 name = "poetry-dynamic-versioning"
-version = "1.4.1"
+version = "1.7.0"
 description = "Plugin for Poetry to enable dynamic versioning based on VCS tags"
 optional = false
 python-versions = "<4.0,>=3.7"
 files = [
-    {file = "poetry_dynamic_versioning-1.4.1-py3-none-any.whl", hash = "sha256:44866ccbf869849d32baed4fc5fadf97f786180d8efa1719c88bf17a471bd663"},
-    {file = "poetry_dynamic_versioning-1.4.1.tar.gz", hash = "sha256:21584d21ca405aa7d83d23d38372e3c11da664a8742995bdd517577e8676d0e1"},
+    {file = "poetry_dynamic_versioning-1.7.0-py3-none-any.whl", hash = "sha256:90db889a43a6da3af43d9609908bd1940c80379509cf92511c9468420a4f1d2f"},
+    {file = "poetry_dynamic_versioning-1.7.0.tar.gz", hash = "sha256:fe9f65e1b408d8b99ea28e5b1edde8f06ca5ff58026aa5d7376ec74f1342a104"},
 ]
 
 [package.dependencies]
@@ -980,7 +980,7 @@ jinja2 = ">=2.11.1,<4"
 tomlkit = ">=0.4"
 
 [package.extras]
-plugin = ["poetry (>=1.2.0,<2.0.0)"]
+plugin = ["poetry (>=1.2.0)"]
 
 [[package]]
 name = "pre-commit"
@@ -1027,13 +1027,13 @@ files = [
 
 [[package]]
 name = "pydantic"
-version = "2.10.4"
+version = "2.10.5"
 description = "Data validation using Python type hints"
 optional = false
 python-versions = ">=3.8"
 files = [
-    {file = "pydantic-2.10.4-py3-none-any.whl", hash = "sha256:597e135ea68be3a37552fb524bc7d0d66dcf93d395acd93a00682f1efcb8ee3d"},
-    {file = "pydantic-2.10.4.tar.gz", hash = "sha256:82f12e9723da6de4fe2ba888b5971157b3be7ad914267dea8f05f82b28254f06"},
+    {file = "pydantic-2.10.5-py3-none-any.whl", hash = "sha256:4dd4e322dbe55472cb7ca7e73f4b63574eecccf2835ffa2af9021ce113c83c53"},
+    {file = "pydantic-2.10.5.tar.gz", hash = "sha256:278b38dbbaec562011d659ee05f63346951b3a248a6f3642e1bc68894ea2b4ff"},
 ]
 
 [package.dependencies]
@@ -1159,13 +1159,13 @@ typing-extensions = ">=4.6.0,<4.7.0 || >4.7.0"
 
 [[package]]
 name = "pygments"
-version = "2.18.0"
+version = "2.19.1"
 description = "Pygments is a syntax highlighting package written in Python."
 optional = false
 python-versions = ">=3.8"
 files = [
-    {file = "pygments-2.18.0-py3-none-any.whl", hash = "sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a"},
-    {file = "pygments-2.18.0.tar.gz", hash = "sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199"},
+    {file = "pygments-2.19.1-py3-none-any.whl", hash = "sha256:9ea1544ad55cecf4b8242fab6dd35a93bbce657034b0611ee383099054ab6d8c"},
+    {file = "pygments-2.19.1.tar.gz", hash = "sha256:61c16d2a8576dc0649d9f39e089b5f02bcd27fba10d8fb4dcc28173f7a45151f"},
 ]
 
 [package.extras]
@@ -1597,13 +1597,13 @@ files = [
 
 [[package]]
 name = "types-pygments"
-version = "2.18.0.20240506"
+version = "2.19.0.20250107"
 description = "Typing stubs for Pygments"
 optional = false
 python-versions = ">=3.8"
 files = [
-    {file = "types-Pygments-2.18.0.20240506.tar.gz", hash = "sha256:4b4c37812c87bbde687dbf27adf5bac593745a321e57f678dbc311571ba2ac9d"},
-    {file = "types_Pygments-2.18.0.20240506-py3-none-any.whl", hash = "sha256:11c90bc1737c9af55e5569558b88df7c2233e12325cb516215f722271444e91d"},
+    {file = "types_Pygments-2.19.0.20250107-py3-none-any.whl", hash = "sha256:34a555ed327f249daed18c6309e6e62770cdb8b9c321029ba7fd852d10b16f10"},
+    {file = "types_pygments-2.19.0.20250107.tar.gz", hash = "sha256:94de72c7f09b956c518f566e056812c698272a7a03a9cd81f0065576c6bd3219"},
 ]
 
 [package.dependencies]


Test patch:
diff --git a/tests/test_dynamic_file_structures.py b/tests/test_dynamic_file_structures.py
new file mode 100644
index 0000000..f86bc8d
--- /dev/null
+++ b/tests/test_dynamic_file_structures.py
@@ -0,0 +1,249 @@
+import warnings
+
+import pytest
+
+import copier
+from copier.errors import MultipleYieldTagsError, YieldTagInFileError
+from tests.helpers import build_file_tree
+
+
+def test_folder_loop(tmp_path_factory: pytest.TempPathFactory) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src
+            / "folder_loop"
+            / "{% yield item from strings %}{{ item }}{% endyield %}"
+            / "{{ item }}.txt.jinja": "Hello {{ item }}",
+        }
+    )
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        copier.run_copy(
+            str(src),
+            dst,
+            data={
+                "strings": ["a", "b", "c"],
+            },
+            defaults=True,
+            overwrite=True,
+        )
+
+        expected_files = [dst / f"folder_loop/{i}/{i}.txt" for i in ["a", "b", "c"]]
+
+        for f in expected_files:
+            assert f.exists()
+            assert f.read_text() == f"Hello {f.parent.name}"
+
+        all_files = [p for p in dst.rglob("*") if p.is_file()]
+        unexpected_files = set(all_files) - set(expected_files)
+
+        assert not unexpected_files, f"Unexpected files found: {unexpected_files}"
+
+
+def test_nested_folder_loop(tmp_path_factory: pytest.TempPathFactory) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src
+            / "nested_folder_loop"
+            / "{% yield string_item from strings %}{{ string_item }}{% endyield %}"
+            / "{% yield integer_item from integers %}{{ integer_item }}{% endyield %}"
+            / "{{ string_item }}_{{ integer_item }}.txt.jinja": "Hello {{ string_item }} {{ integer_item }}",
+        }
+    )
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        copier.run_copy(
+            str(src),
+            dst,
+            data={
+                "strings": ["a", "b"],
+                "integers": [1, 2, 3],
+            },
+            defaults=True,
+            overwrite=True,
+        )
+
+        expected_files = [
+            dst / f"nested_folder_loop/{s}/{i}/{s}_{i}.txt"
+            for s in ["a", "b"]
+            for i in [1, 2, 3]
+        ]
+
+        for f in expected_files:
+            assert f.exists()
+            assert f.read_text() == f"Hello {f.parent.parent.name} {f.parent.name}"
+
+        all_files = [p for p in dst.rglob("*") if p.is_file()]
+        unexpected_files = set(all_files) - set(expected_files)
+
+        assert not unexpected_files, f"Unexpected files found: {unexpected_files}"
+
+
+def test_file_loop(tmp_path_factory: pytest.TempPathFactory) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src
+            / "file_loop"
+            / "{% yield string_item from strings %}{{ string_item }}{% endyield %}.jinja": "Hello {{ string_item }}",
+        }
+    )
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        copier.run_copy(
+            str(src),
+            dst,
+            data={
+                "strings": [
+                    "a.txt",
+                    "b.txt",
+                    "c.txt",
+                    "",
+                ],  # if rendred as '.jinja', it will not be created
+            },
+            defaults=True,
+            overwrite=True,
+        )
+
+        expected_files = [dst / f"file_loop/{i}.txt" for i in ["a", "b", "c"]]
+        for f in expected_files:
+            assert f.exists()
+            assert f.read_text() == f"Hello {f.stem}.txt"
+
+        all_files = [p for p in dst.rglob("*") if p.is_file()]
+        unexpected_files = set(all_files) - set(expected_files)
+
+        assert not unexpected_files, f"Unexpected files found: {unexpected_files}"
+
+
+def test_folder_loop_dict_items(tmp_path_factory: pytest.TempPathFactory) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src
+            / "folder_loop_dict_items"
+            / "{% yield dict_item from dicts %}{{ dict_item.folder_name }}{% endyield %}"
+            / "{{ dict_item.file_name }}.txt.jinja": "Hello {{ '-'.join(dict_item.content) }}",
+        }
+    )
+
+    dicts = [
+        {
+            "folder_name": "folder_a",
+            "file_name": "file_a",
+            "content": ["folder_a", "file_a"],
+        },
+        {
+            "folder_name": "folder_b",
+            "file_name": "file_b",
+            "content": ["folder_b", "file_b"],
+        },
+        {
+            "folder_name": "folder_c",
+            "file_name": "file_c",
+            "content": ["folder_c", "file_c"],
+        },
+    ]
+
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+
+        copier.run_copy(
+            str(src),
+            dst,
+            data={"dicts": dicts},
+            defaults=True,
+            overwrite=True,
+        )
+
+        expected_files = [
+            dst / f"folder_loop_dict_items/{d['folder_name']}/{d['file_name']}.txt"
+            for d in [
+                {"folder_name": "folder_a", "file_name": "file_a"},
+                {"folder_name": "folder_b", "file_name": "file_b"},
+                {"folder_name": "folder_c", "file_name": "file_c"},
+            ]
+        ]
+
+        for f in expected_files:
+            assert f.exists()
+            assert f.read_text() == f"Hello {'-'.join([f.parts[-2], f.stem])}"
+
+        all_files = [p for p in dst.rglob("*") if p.is_file()]
+        unexpected_files = set(all_files) - set(expected_files)
+
+        assert not unexpected_files, f"Unexpected files found: {unexpected_files}"
+
+
+def test_raise_yield_tag_in_file(tmp_path_factory: pytest.TempPathFactory) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src
+            / "file.txt.jinja": "{% yield item from strings %}{{ item }}{% endyield %}",
+        }
+    )
+
+    with pytest.raises(YieldTagInFileError, match="file.txt.jinja"):
+        copier.run_copy(
+            str(src),
+            dst,
+            data={
+                "strings": ["a", "b", "c"],
+            },
+            defaults=True,
+            overwrite=True,
+        )
+
+
+def test_raise_multiple_yield_tags(tmp_path_factory: pytest.TempPathFactory) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    # multiple yield tags, not nested
+    file_name = "{% yield item1 from strings %}{{ item1 }}{% endyield %}{% yield item2 from strings %}{{ item2 }}{% endyield %}"
+
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src / file_name: "",
+        }
+    )
+
+    with pytest.raises(MultipleYieldTagsError, match="item"):
+        copier.run_copy(
+            str(src),
+            dst,
+            data={
+                "strings": ["a", "b", "c"],
+            },
+            defaults=True,
+            overwrite=True,
+        )
+
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    # multiple yield tags, nested
+    file_name = "{% yield item1 from strings %}{% yield item2 from strings %}{{ item1 }}{{ item2 }}{% endyield %}{% endyield %}"
+
+    build_file_tree(
+        {
+            src / "copier.yml": "",
+            src / file_name: "",
+        }
+    )
+
+    with pytest.raises(MultipleYieldTagsError, match="item"):
+        copier.run_copy(
+            str(src),
+            dst,
+            data={
+                "strings": ["a", "b", "c"],
+            },
+            defaults=True,
+            overwrite=True,
+        )