Repository: copier-org/copier
Pull request #1813 (2024-10-11 14:48:28)

fix: Don't mark files without conflict markers as unmerged
Closes #1763.

**Please squash when merging, do not rebase.**

pawamoy: Once again having trouble with Nix :sweat_smile: 

```
error: creating directory '/nix/store': Permission denied
error: creating directory '/nix/store': Permission denied
```
pawamoy: OK now I just have to find out how to write a test for this :melting_face: 
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/copier-org/copier/pull/1813?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 97.43%. Comparing base [(`4ce9518`)](https://app.codecov.io/gh/copier-org/copier/commit/4ce9518472ec858e5622e9cafccaf2ab23f7be1e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) to head [(`973eae8`)](https://app.codecov.io/gh/copier-org/copier/commit/973eae85ee548397d3795008c668c4842b23c965?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1813      +/-   ##
==========================================
- Coverage   97.78%   97.43%   -0.36%     
==========================================
  Files          49       49              
  Lines        5108     5110       +2     
==========================================
- Hits         4995     4979      -16     
- Misses        113      131      +18     
```

| [Flag](https://app.codecov.io/gh/copier-org/copier/pull/1813/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) | Coverage Δ | |
|---|---|---|
| [unittests](https://app.codecov.io/gh/copier-org/copier/pull/1813/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org) | `97.43% <100.00%> (-0.36%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/copier-org/copier/pull/1813?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=copier-org).

lkubb: > OK now I just have to find out how to write a test for this 🫠

Doing the same change in the template's new version and a generated project before updating should be enough to trigger this bug I think.
lkubb: @pawamoy It doesn't pass when I run it on latest master:

```
====================================================== FAILURES ======================================================
__________________________ test_3way_merged_files_without_conflicts_are_not_marked_unmerged __________________________
[...]
        with local.cwd(dst):
            lines = git("status", "--porcelain=v1").strip().splitlines()
>           assert not any(
                line.startswith("UU") and normalize_git_path(line[3:]) == filename
                for line in lines
            )
E           assert not True
E            +  where True = any(<generator object test_3way_merged_files_without_conflicts_are_not_marked_unmerged.<locals>.<genexpr> at 0x107543a00>)

tests/test_updatediff.py:91: AssertionError
```

pawamoy: Thanks for testing! Weird, can't make it fail locally. Have commented the fix to see how the test behaves in CI.
lkubb: You need to add back what the patch replaced when commenting it out, otherwise the test will pass:

```
conflicted.append(fname)
```
pawamoy: OMG of course. Thanks :melting_face: 
pawamoy: Properly failing locally without the fix. Ready for review then! Thanks @lkubb, really appreciate the speedy help!
pawamoy: Codecov doesn't like me. Here it says -0.11% and when I click on their link it says +0.02%...

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e097b52..b5d795d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. This projec
 adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/) versioning schema, and
 the changelog itself conforms to [Keep A Changelog](https://keepachangelog.com/).
 
+## v9.4.1 (2024-10-18)
+
+### Fix
+
+-   restore support for `preserve_symlinks: false` for directories (#1820)
+
 ## v9.4.0 (2024-10-15)
 
 ### Fix
diff --git a/README.md b/README.md
index d81f3c4..9886b09 100644
--- a/README.md
+++ b/README.md
@@ -9,6 +9,7 @@
 [![PyPI](https://img.shields.io/pypi/v/copier?logo=pypi&logoColor=%23959DA5)](https://pypi.org/project/copier/)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![Documentation Status](https://img.shields.io/readthedocs/copier/latest?logo=readthedocs)](https://copier.readthedocs.io/en/latest)
+[![](https://img.shields.io/badge/Gurubase-Ask%20Copier%20Guru-006BFF)](https://gurubase.io/g/copier)
 
 A library and CLI app for rendering project templates.
 
diff --git a/copier/main.py b/copier/main.py
index 69171bd..7f38bda 100644
--- a/copier/main.py
+++ b/copier/main.py
@@ -1105,8 +1105,15 @@ class Worker:
                         )
                         # Remove rejection witness
                         Path(f"{fname}.rej").unlink()
-                        # Store file name for marking it as unmerged after the loop
-                        conflicted.append(fname)
+                        # The 3-way merge might have resolved conflicts automatically,
+                        # so we need to check if the file contains conflict markers
+                        # before storing the file name for marking it as unmerged after the loop.
+                        with open(fname) as conflicts_candidate:
+                            if any(
+                                line.rstrip() in {"<<<<<<< before updating", ">>>>>>> after updating"}
+                                for line in conflicts_candidate
+                            ):
+                                conflicted.append(fname)
                     # Forcefully mark files with conflict markers as unmerged,
                     # see SO post: https://stackoverflow.com/questions/77391627/
                     # and Git docs: https://git-scm.com/docs/git-update-index#_using_index_info.
diff --git a/copier/user_data.py b/copier/user_data.py
index 0a2af77..85698fa 100644
--- a/copier/user_data.py
+++ b/copier/user_data.py
@@ -177,7 +177,7 @@ class Question:
     var_name: str
     answers: AnswersMap
     jinja_env: SandboxedEnvironment
-    choices: Sequence[Any] | dict[Any, Any] = field(default_factory=list)
+    choices: Sequence[Any] | dict[Any, Any] | str = field(default_factory=list)
     multiselect: bool = False
     default: Any = MISSING
     help: str = ""
@@ -291,8 +291,10 @@ class Question:
         result = []
         choices = self.choices
         default = self.get_default()
-        if isinstance(self.choices, dict):
-            choices = list(self.choices.items())
+        if isinstance(choices, str):
+            choices = parse_yaml_string(self.render_value(self.choices))
+        if isinstance(choices, dict):
+            choices = list(choices.items())
         for choice in choices:
             # If a choice is a value pair
             if isinstance(choice, (tuple, list)):
diff --git a/docs/configuring.md b/docs/configuring.md
index 13c2d29..636f04e 100644
--- a/docs/configuring.md
+++ b/docs/configuring.md
@@ -116,6 +116,41 @@ Supported keys:
         the message is shown. Choice validation is useful when the validity of a choice
         depends on the answer to a previous question.
 
+    !!! tip "Dynamic choices"
+
+        Choices can be created dynamically by using a templated string which renders
+        as valid list-style, dict-style, or tuple-style choices in YAML format. For
+        example:
+
+        ```yaml title="copier.yml"
+        language:
+            type: str
+            help: Which programming language do you use?
+            choices:
+                - python
+                - node
+
+        dependency_manager:
+            type: str
+            help: Which dependency manager do you use?
+            choices: |
+                {%- if language == "python" %}
+                - poetry
+                - pipenv
+                {%- else %}
+                - npm
+                - yarn
+                {%- endif %}
+        ```
+
+        Dynamic choices can be used as an alternative approach to conditional choices
+        via validators where dynamic choices hide disabled choices whereas choices
+        disabled via validators are visible with along with the validator's error
+        message but cannot be selected.
+
+        When combining dynamic choices with validators, make sure to escape the
+        validator template using `{% raw %}...{% endraw %}`.
+
     !!! warning
 
         You are able to use different types for each choice value, but it is not
diff --git a/pyproject.toml b/pyproject.toml
index 2fb505f..601fcec 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -168,7 +168,7 @@ annotated_tag = true
 changelog_incremental = true
 tag_format = "v$version"
 update_changelog_on_bump = true
-version = "9.4.0"
+version = "9.4.1"
 
 [build-system]
 requires = ["poetry-core>=1.7.0", "poetry-dynamic-versioning>=1.1.0"]


Test patch:
diff --git a/tests/test_copy.py b/tests/test_copy.py
index 0717a61..c98060a 100644
--- a/tests/test_copy.py
+++ b/tests/test_copy.py
@@ -9,6 +9,7 @@ from contextlib import nullcontext as does_not_raise
 from decimal import Decimal
 from enum import Enum
 from pathlib import Path
+from textwrap import dedent, indent
 from time import sleep
 from typing import Any, ContextManager
 
@@ -963,3 +964,81 @@ def test_multiselect_choices_preserve_order(
     )
     copier.run_copy(str(src), dst, data={"q": ["three", "one", "two"]})
     assert yaml.safe_load((dst / "q.yml").read_text()) == ["one", "two", "three"]
+
+
+@pytest.mark.parametrize(
+    "spec",
+    [
+        (
+            """\
+            choices: '["one", "two", "three"]'
+            """
+        ),
+        (
+            """\
+            choices: '[{% for c in ["one", "two", "three"] %}{{ c }}{% if not loop.last %},{% endif %}{% endfor %}]'
+            """
+        ),
+        (
+            """\
+            choices: |
+                {%- for c in ['one', 'two', 'three'] %}
+                - {{ c }}
+                {%- endfor %}
+            """
+        ),
+        (
+            """\
+            choices: |
+                {%- for c in ['one', 'two', 'three'] %}
+                {{ c }}: null
+                {%- endfor %}
+            """
+        ),
+        (
+            """\
+            choices: |
+                {%- for c in ['one', 'two', 'three'] %}
+                {{ c|upper }}: {{ c }}
+                {%- endfor %}
+            """
+        ),
+        (
+            """\
+            choices: |
+                {%- for c in ['one', 'two', 'three'] %}
+                - [{{ c|upper }}, {{ c }}]
+                {%- endfor %}
+            """
+        ),
+        (
+            """\
+            choices: |
+                {%- for c in ['one', 'two', 'three'] %}
+                {{ c|upper }}:
+                    value: {{ c }}
+                {%- endfor %}
+            """
+        ),
+        (
+            """\
+            choices: |
+                {%- for c in ['one', 'two', 'three'] %}
+                {{ c|upper }}:
+                    value: {{ c }}
+                    validator: "{% raw %}{% if false %}Validation error{% endif %}{% endraw %}"
+                {%- endfor %}
+            """
+        ),
+    ],
+)
+def test_templated_choices(tmp_path_factory: pytest.TempPathFactory, spec: str) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            (src / "copier.yml"): f"q:\n    type: str\n{indent(dedent(spec), ' ' * 4)}",
+            (src / "q.txt.jinja"): "{{ q }}",
+        }
+    )
+    copier.run_copy(str(src), dst, data={"q": "two"})
+    assert yaml.safe_load((dst / "q.txt").read_text()) == "two"
diff --git a/tests/test_templated_prompt.py b/tests/test_templated_prompt.py
index c69c49c..d7e745d 100644
--- a/tests/test_templated_prompt.py
+++ b/tests/test_templated_prompt.py
@@ -368,6 +368,63 @@ def test_templated_prompt_with_conditional_choices(
     tui.sendline()
 
 
+@pytest.mark.parametrize(
+    "cloud, iac_choices",
+    [
+        ("Any", ["Terraform"]),
+        ("AWS", ["Terraform", "Cloud Formation"]),
+        ("Azure", ["Terraform", "Azure Resource Manager"]),
+        ("GCP", ["Terraform", "Deployment Manager"]),
+    ],
+)
+def test_templated_prompt_with_templated_choices(
+    tmp_path_factory: pytest.TempPathFactory,
+    spawn: Spawn,
+    cloud: str,
+    iac_choices: str,
+) -> None:
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+    build_file_tree(
+        {
+            (src / "copier.yml"): (
+                """\
+                cloud:
+                    type: str
+                    help: Which cloud provider do you use?
+                    choices:
+                        - Any
+                        - AWS
+                        - Azure
+                        - GCP
+
+                iac:
+                    type: str
+                    help: Which IaC tool do you use?
+                    choices: |
+                        Terraform: tf
+                        {%- if cloud == 'AWS' %}
+                        Cloud Formation: cf
+                        {%- endif %}
+                        {%- if cloud == 'Azure' %}
+                        Azure Resource Manager: arm
+                        {%- endif %}
+                        {%- if cloud == 'GCP' %}
+                        Deployment Manager: dm
+                        {%- endif %}
+                """
+            ),
+        }
+    )
+    tui = spawn(
+        COPIER_PATH + ("copy", f"--data=cloud={cloud}", str(src), str(dst)),
+        timeout=10,
+    )
+    expect_prompt(tui, "iac", "str", help="Which IaC tool do you use?")
+    for iac in iac_choices:
+        tui.expect_exact(iac)
+    tui.sendline()
+
+
 def test_templated_prompt_update_previous_answer_disabled(
     tmp_path_factory: pytest.TempPathFactory, spawn: Spawn
 ) -> None:
diff --git a/tests/test_updatediff.py b/tests/test_updatediff.py
index a556de0..73774e4 100644
--- a/tests/test_updatediff.py
+++ b/tests/test_updatediff.py
@@ -1239,7 +1239,7 @@ def test_conflicted_files_are_marked_unmerged(
     run_update(dst_path=dst, defaults=True, overwrite=True, conflict="inline")
     assert "_commit: v2" in (dst / ".copier-answers.yml").read_text()
 
-    # Assert that the file still exists, has inline markers,
+    # Assert that the file still exists, has inline conflict markers,
     # and is reported as "unmerged" by Git.
     assert (dst / filename).exists()
 
@@ -1263,6 +1263,72 @@ def test_conflicted_files_are_marked_unmerged(
         )
 
 
+def test_3way_merged_files_without_conflicts_are_not_marked_unmerged(
+    tmp_path_factory: pytest.TempPathFactory,
+) -> None:
+    filename = "readme.md"
+
+    # Template in v1 has a file with a single line;
+    # in v2 it changes that line.
+    # Meanwhile, downstream project made the same change.
+    src, dst = map(tmp_path_factory.mktemp, ("src", "dst"))
+
+    # First, create the template with an initial file
+    build_file_tree(
+        {
+            (src / filename): "upstream version 1",
+            (src / "{{_copier_conf.answers_file}}.jinja"): (
+                "{{_copier_answers|to_nice_yaml}}"
+            ),
+        }
+    )
+    with local.cwd(src):
+        git_init("hello template")
+        git("tag", "v1")
+
+    # Generate the project a first time, assert the file exists
+    run_copy(str(src), dst, defaults=True, overwrite=True)
+    assert (dst / filename).exists()
+    assert "_commit: v1" in (dst / ".copier-answers.yml").read_text()
+
+    # Start versioning the generated project
+    with local.cwd(dst):
+        git_init("hello project")
+
+        # After first commit, change the file, commit again
+        Path(filename).write_text("upstream version 2")
+        git("commit", "-am", "updated file")
+
+    # Now change the template
+    with local.cwd(src):
+        # Update the file
+        Path(filename).write_text("upstream version 2")
+
+        # Commit the changes
+        git("add", ".", "-A")
+        git("commit", "-m", "change line in file")
+        git("tag", "v2")
+
+    # Finally, update the generated project
+    run_update(dst_path=dst, defaults=True, overwrite=True, conflict="inline")
+    assert "_commit: v2" in (dst / ".copier-answers.yml").read_text()
+
+    # Assert that the file still exists, does not have inline conflict markers,
+    # and is not reported as "unmerged" by Git.
+    assert (dst / filename).exists()
+
+    expected_contents = "upstream version 2"
+    assert (dst / filename).read_text() == expected_contents
+    assert not (dst / f"{filename}.rej").exists()
+
+    with local.cwd(dst):
+        lines = git("status", "--porcelain=v1").strip().splitlines()
+        assert not any(
+            line.startswith("UU") and normalize_git_path(line[3:]) == filename
+            for line in lines
+        )
+
+
 def test_update_with_new_file_in_template_and_project(
     tmp_path_factory: pytest.TempPathFactory,
 ) -> None: