Repository: meltano/meltano

Issue:
feature: support glob patterns for paths in file bundle plugin `update` extra
### Feature scope

CLI (options, error messages, logging, etc.)

### Description

### Overview
File bundle plugins can specify a number of files to update with the `update` extra, when `meltano upgrade files` is run.

`meltano.yml`
```yml
version: 1
default_environment: dev
environments:
- name: dev
- name: staging
- name: prod
project_id: fefc3baf-ebb0-4f68-87d1-fe5b3afbe6e8
plugins:
  files:
  - name: files-dbt
    pip_url: git+https://github.com/meltano/files-dbt
    update:
      transform/models/.gitkeep: true
      transform/profile/profiles.yml: true
      transform/.gitignore: true
      transform/dbt_project.yml: true
```

Currently, each file than can be updated by `meltano upgrade files` must have its path declared under `update` individually. This can lead to bloated `meltano.yml` file bundle definitions that specify many files within common directories as upgradable.

### Proposal
Support glob patterns to reduce the number of paths needed to specify all files required for upgrade.

All bundle files:
```yml
    update:
      '**/*': true
```

All bundle `.yml` files:
```yml
    update:
      '**/*.yml': true
```

All bundle `.yml` files under the `transform` directory:
```yml
    update:
      'transform/**/*.yml': true
```


Discussion:

Resolving patch:
diff --git a/src/meltano/core/plugin/file.py b/src/meltano/core/plugin/file.py
--- a/src/meltano/core/plugin/file.py
+++ b/src/meltano/core/plugin/file.py
@@ -107,7 +107,7 @@ def project_file_contents(
         """
 
         def with_update_header(content: str, relative_path: PathLike):
-            if str(relative_path) in paths_to_update:
+            if any(relative_path.match(path) for path in paths_to_update):
                 content = "\n\n".join([self.update_file_header(relative_path), content])
 
             return content
@@ -209,12 +209,11 @@ def files_to_update(
         Returns:
             A dictionary of file names and their contents.
         """
+        file_contents = self.project_file_contents(project, paths_to_update)
         return {
             relative_path: content
-            for relative_path, content in self.project_file_contents(
-                project, paths_to_update
-            ).items()
-            if str(relative_path) in paths_to_update
+            for relative_path, content in file_contents.items()
+            if any(relative_path.match(path) for path in paths_to_update)
         }
 
     def create_files(


Test patch:
diff --git a/tests/meltano/cli/test_upgrade.py b/tests/meltano/cli/test_upgrade.py
--- a/tests/meltano/cli/test_upgrade.py
+++ b/tests/meltano/cli/test_upgrade.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import json
 import platform
 import shutil
 
@@ -155,6 +156,49 @@ def test_upgrade_files(
         assert_cli_runner(result)
         assert "Updated orchestrate/dags/meltano.py" in output
 
+    def test_upgrade_files_glob_path(
+        self, session, project, cli_runner, config_service, meltano_hub_service
+    ):
+        if platform.system() == "Windows":
+            pytest.xfail(
+                "Doesn't pass on windows, this is currently being tracked here https://github.com/meltano/meltano/issues/3444"
+            )
+
+        with mock.patch(
+            "meltano.core.project_plugins_service.MeltanoHubService",
+            return_value=meltano_hub_service,
+        ):
+            result = cli_runner.invoke(cli, ["add", "files", "airflow"])
+        assert_cli_runner(result)
+
+        file_path = project.root_dir("orchestrate/dags/meltano.py")
+        file_path.write_text("Overwritten!")
+
+        # override airflow--meltano.lock update extra config
+        result = cli_runner.invoke(
+            cli,
+            [
+                "config",
+                "--plugin-type",
+                "files",
+                "airflow",
+                "set",
+                "_update",
+                json.dumps(
+                    {
+                        "orchestrate/dags/meltano.py": False,
+                        "*.py": True,
+                    },
+                ),
+            ],
+        )
+        assert_cli_runner(result)
+
+        result = cli_runner.invoke(cli, ["upgrade", "files"])
+        output = result.stdout + result.stderr
+        assert_cli_runner(result)
+        assert "Updated orchestrate/dags/meltano.py" in output
+
     def test_upgrade_database(self, project, cli_runner):
         result = cli_runner.invoke(cli, ["upgrade", "database"])
         assert_cli_runner(result)