internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: meltano/meltano

Issue:
Verify `meltano install` telemetry fires for malformed plugin entries
In #6109 @pnadolny13 noticed that with the following entry no events where fired:

```yaml
  - name: tap-something-invalid
    variant: meltanolabs
    pip_url: git+https://github.com/foo/tap-something-invalid.git
```

I wasn't able to reproduce that at the time and did see two events (started/aborted) come across. We should double check though, its entirely possible that my local dev setup had a seperate issue that was triggering the `aborted` event.


Discussion:

Resolving patch:
diff --git a/src/meltano/cli/install.py b/src/meltano/cli/install.py
--- a/src/meltano/cli/install.py
+++ b/src/meltano/cli/install.py
@@ -5,7 +5,6 @@
 
 from meltano.core.legacy_tracking import LegacyTracker
 from meltano.core.plugin import PluginType
-from meltano.core.plugin.error import PluginNotFoundError
 from meltano.core.project_plugins_service import ProjectPluginsService
 from meltano.core.tracking import PluginsTrackingContext, Tracker
 from meltano.core.tracking import cli as cli_tracking
@@ -52,24 +51,18 @@ def install(project, plugin_type, plugin_name, clean, parallelism):
 
     plugins_service = ProjectPluginsService(project)
 
-    if plugin_type:
-        try:
+    try:
+        if plugin_type:
             plugin_type = PluginType.from_cli_argument(plugin_type)
-        except ValueError:
-            # if we fail because plugin_type is not valid we have no plugins to instrument
-            tracker.track_command_event(cli_tracking.STARTED)
-            tracker.track_command_event(cli_tracking.ABORTED)
-            raise
-        plugins = plugins_service.get_plugins_of_type(plugin_type)
-        if plugin_name:
-            plugins = [plugin for plugin in plugins if plugin.name in plugin_name]
-    else:
-        try:
+            plugins = plugins_service.get_plugins_of_type(plugin_type)
+            if plugin_name:
+                plugins = [plugin for plugin in plugins if plugin.name in plugin_name]
+        else:
             plugins = list(plugins_service.plugins())
-        except PluginNotFoundError:
-            tracker.track_command_event(cli_tracking.STARTED)
-            tracker.track_command_event(cli_tracking.ABORTED)
-            raise
+    except Exception:
+        tracker.track_command_event(cli_tracking.STARTED)
+        tracker.track_command_event(cli_tracking.ABORTED)
+        raise
 
     click.echo(f"Installing {len(plugins)} plugins...")
     tracker.add_contexts(


Test patch:
Repository: meltano/meltano

Issue:
Address warning in Airflow plugin version check
> not super urgent, but as we move into supporting newer Python versions 
> (https://github.com/meltano/meltano/pull/6135) and bumping Meltano's dependencies (https://github.com/meltano/meltano/issues/6264), we might break Airflow support.
> 
> It's also probably a very quick (< 1 hour) fix by replacing distutils.StrictVersion with the packaging.version equivalent.
> 

```
src/meltano/core/plugin/airflow.py:110: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if StrictVersion(version) < StrictVersion("2.0.0")
```


Discussion:
@edgarrmondragon what's the solution here? And how urgent is this?

Resolving patch:
diff --git a/src/meltano/core/plugin/airflow.py b/src/meltano/core/plugin/airflow.py
--- a/src/meltano/core/plugin/airflow.py
+++ b/src/meltano/core/plugin/airflow.py
@@ -3,7 +3,8 @@
 import logging
 import os
 import subprocess
-from distutils.version import StrictVersion
+
+from packaging.version import Version
 
 from meltano.core.behavior.hookable import hook
 from meltano.core.error import AsyncSubprocessError
@@ -146,9 +147,7 @@ async def before_configure(self, invoker: AirflowInvoker, session):  # noqa: WPS
 
         version = stdout.decode()
         init_db_cmd = (
-            ["initdb"]
-            if StrictVersion(version) < StrictVersion("2.0.0")
-            else ["db", "init"]
+            ["initdb"] if Version(version) < Version("2.0.0") else ["db", "init"]
         )
 
         handle = await invoker.invoke_async(


Test patch:
Repository: meltano/meltano

Issue:
`meltano select tap-gitlab tags "*"` fails on Windows
Example: https://gitlab.com/meltano/legacy-ci/meltano/-/jobs/2653845782

This occurs with Meltano built from `main`, i.e. `fb6ed89799eb341bea295350ea809e5049ee7ca6`.


Discussion:
Found the root cause: https://github.com/pallets/click/pull/1830

Resolving patch:
diff --git a/src/meltano/cli/cli.py b/src/meltano/cli/cli.py
--- a/src/meltano/cli/cli.py
+++ b/src/meltano/cli/cli.py
@@ -1,6 +1,6 @@
 import logging  # noqa: D100
 import sys
-import warnings  # noqa: F401
+from typing import NoReturn
 
 import click
 
@@ -13,7 +13,28 @@
 logger = logging.getLogger(__name__)
 
 
-@click.group(invoke_without_command=True, no_args_is_help=True)
+class NoWindowsGlobbingGroup(click.Group):
+    """A Click group that does not perform glob expansion on Windows.
+
+    This restores the behaviour of Click's globbing to how it was before v8.
+    Click (as of version 8.1.3) ignores quotes around an asterisk, which makes
+    it behave differently than most shells that support globbing, and make some
+    typical Meltano commands fail, e.g. `meltano select tap-gitlab tags "*"`.
+    """
+
+    def main(self, *args, **kwargs) -> NoReturn:
+        """Invoke the Click CLI with Windows globbing disabled.
+
+        Parameters:
+            args: Positional arguments for the Click group.
+            kwargs: Keyword arguments for the Click group.
+        """
+        return super().main(*args, windows_expand_args=False, **kwargs)
+
+
+@click.group(
+    cls=NoWindowsGlobbingGroup, invoke_without_command=True, no_args_is_help=True
+)
 @click.option("--log-level", type=click.Choice(LEVELS.keys()))
 @click.option(
     "--log-config", type=str, help="Path to a python logging yaml config file."


Test patch:
Repository: meltano/meltano

Issue:
`meltano --help` fails on Windows in some environments
Running any Meltano command in my fresh Windows VM results in the following error:

![image](https://user-images.githubusercontent.com/11428666/176504379-a9489721-09b5-4a2d-ad83-c2f5a37adfb0.png)

Sorry for the image of text - I forgot to enable the shared clipboard for the VM.

The failure is happening here:

```python
        freedesktop_data = (
            platform.freedesktop_os_release()
            if safe_hasattr(platform, "freedesktop_os_release")
            else defaultdict(type(None))
        )
```

This might be because I'm running under git bash instead of Powershell.

We had assumed that if `freedesktop_os_release` is present, it will work. This is apparently only sometimes true. To prevent problems for Windows users we should put the call to `platform.freedesktop_os_release` in a try-except block. Doing so will also mean we no longer need to call `hasattr`.


Discussion:
Thanks for identifying and logging, @WillDaSilva . Would be worth picking up as soon as we have capacity. Thanks!
Okay @aaronsteers, I've moved it into the current iteration, and moved https://github.com/meltano/meltano/issues/6234 into the next one to compensate (not that they'll take nearly the same amount of time, but we'll see what I have time for in the end).

Resolving patch:
diff --git a/src/meltano/core/tracking/contexts/environment.py b/src/meltano/core/tracking/contexts/environment.py
--- a/src/meltano/core/tracking/contexts/environment.py
+++ b/src/meltano/core/tracking/contexts/environment.py
@@ -56,11 +56,10 @@ def system_info(self) -> dict[str, Any]:
         Returns:
             A dictionary containing system information.
         """
-        freedesktop_data = (
-            platform.freedesktop_os_release()
-            if safe_hasattr(platform, "freedesktop_os_release")
-            else defaultdict(type(None))
-        )
+        try:
+            freedesktop_data = platform.freedesktop_os_release()
+        except Exception:
+            freedesktop_data = defaultdict(type(None))
 
         return {
             "system_name": platform.system() or None,


Test patch:
Repository: meltano/meltano

Issue:
Uvicorn Windows message points to wrong flag
```2022-06-30T19:52:16.704167Z [warning  ] Add ff.start_uvicorn: True to your meltano.yml to supress this waring```

Should point to

ff.enable_uvicorn [env: MELTANO_FF_ENABLE_UVICORN] current value: False (default)

Should be an easy one!


Discussion:
Oh wow sorry about that.  I know where that is.  I will create a PR for it
> Oh wow sorry about that. I know where that is. I will create a PR for it

Don't worry about it! I'm just filing the issues.

It's amazing Uvicorn is working on Window it's so much better than where we were. This is all behind a feature flag as well so that's good news I think

Resolving patch:
diff --git a/src/meltano/api/workers/api_worker.py b/src/meltano/api/workers/api_worker.py
--- a/src/meltano/api/workers/api_worker.py
+++ b/src/meltano/api/workers/api_worker.py
@@ -44,7 +44,7 @@ def run(self):
                         "Windows OS detected auto setting ff.enable_uvicorn"
                     )
                     logging.warning(
-                        "Add ff.start_uvicorn: True to your meltano.yml to supress this waring"
+                        "Add ff.enable_uvicorn: True to your meltano.yml to supress this waring"
                     )
                     enable_uvicorn = True
 


Test patch:
Repository: meltano/meltano

Issue:
Airflow docs incorrect for adding files
This was raised in slack https://meltano.slack.com/archives/C01TCRBBJD7/p1655231194958339

The docs say `meltano add files airflow` but now we prefix the file bundles with `files-` so it should be updated to `meltano add files files-airflow`.

Its probably worth while to search for other references that might be incorrect also. The file bundle readme's look like they could use an update as well https://github.com/meltano/files-airflow.


Discussion:

Resolving patch:
diff --git a/src/meltano/core/plugin/dbt/base.py b/src/meltano/core/plugin/dbt/base.py
--- a/src/meltano/core/plugin/dbt/base.py
+++ b/src/meltano/core/plugin/dbt/base.py
@@ -63,7 +63,7 @@ async def install(self, reason, clean):
         except FileNotFoundError as err:
             relative_path = Path(err.filename).relative_to(self.project.root)
             raise PluginInstallError(
-                f"File {relative_path}' could not be found. Run `meltano add files dbt` to set up a dbt project."
+                f"File {relative_path}' could not be found. Run `meltano add files files-dbt` to set up a dbt project."
             )
 
 


Test patch:
Repository: meltano/meltano

Issue:
Remove run 'preview' references
As of v2.0.0, `meltano run` is no longer a preview feature.  We can therefore remove the remaining references to `meltano run` being in 'preview' from the CLI.

<img width="762" alt="Screenshot 2022-07-05 at 10 49 34" src="https://user-images.githubusercontent.com/5585874/177345173-62a09d70-b72b-49ef-b644-a6d16275394f.png">


Discussion:

Resolving patch:
diff --git a/src/meltano/cli/run.py b/src/meltano/cli/run.py
--- a/src/meltano/cli/run.py
+++ b/src/meltano/cli/run.py
@@ -23,9 +23,7 @@
 logger = structlog.getLogger(__name__)
 
 
-@cli.command(
-    cls=InstrumentedCmd, short_help="[preview] Run a set of plugins in series."
-)
+@cli.command(cls=InstrumentedCmd, short_help="Run a set of plugins in series.")
 @click.option(
     "--dry-run",
     help="Do not run, just parse the invocation, validate it, and explain what would be executed.",
@@ -70,7 +68,7 @@ async def run(
     `meltano run some_extractor some_loader some_plugin:some_command` and are run in the order they are specified
     from left to right. A failure in any block will cause the entire run to abort.
 
-    Multiple commmand blocks can be chained together or repeated, and tap/target pairs will automatically be linked:
+    Multiple command blocks can be chained together or repeated, and tap/target pairs will automatically be linked:
 
         `meltano run tap-gitlab target-postgres dbt:test dbt:run`\n
         `meltano run tap-gitlab target-postgres tap-salesforce target-mysql ...`\n
@@ -83,8 +81,6 @@ async def run(
 
     The above command will create two jobs with state IDs `prod:tap-gitlab-to-target-postgres` and `prod:tap-salesforce-to-target-mysql`.
 
-    This a preview feature - its functionality and cli signature is still evolving.
-
     \b\nRead more at https://docs.meltano.com/reference/command-line-interface#run
     """
     if dry_run:


Test patch:
Repository: meltano/meltano

Issue:
Pytest warnings about unknown feature flag `strict_env_var_mode`
Example: https://github.com/meltano/meltano/runs/7217473356

```
=============================== warnings summary ===============================
tests/meltano/cli/test_add.py: 4 warnings
tests/meltano/cli/test_config.py: 4 warnings
tests/meltano/cli/test_elt.py: 16 warnings
tests/meltano/cli/test_invoke.py: 8 warnings
tests/meltano/cli/test_run.py: 12 warnings
tests/meltano/cli/test_select.py: 1 warning
tests/meltano/cli/test_upgrade.py: 1 warning
tests/meltano/core/test_environment_variables.py: 10 warnings
tests/meltano/core/test_plugin_invoker.py: 2 warnings
tests/meltano/core/plugin/test_plugin_settings.py: 1 warning
  /home/runner/work/meltano/meltano/src/meltano/core/settings_service.py:422: RuntimeWarning: Unknown setting 'ff.strict_env_var_mode' - the default value `None` will be used
    warn(
```



Discussion:

Resolving patch:
diff --git a/src/meltano/core/settings_service.py b/src/meltano/core/settings_service.py
--- a/src/meltano/core/settings_service.py
+++ b/src/meltano/core/settings_service.py
@@ -3,11 +3,11 @@
 
 import logging
 import os
+import warnings
 from abc import ABC, abstractmethod
 from contextlib import contextmanager
 from enum import Enum
 from typing import Generator, Iterable
-from warnings import warn
 
 from meltano.core.project import Project
 from meltano.core.utils import expand_env_vars as do_expand_env_vars
@@ -419,7 +419,7 @@ def get_with_metadata(  # noqa: WPS210, WPS615
         self.log(f"Got setting {name!r} with metadata: {metadata}")
 
         if setting_def is None and metadata["source"] is SettingValueStore.DEFAULT:
-            warn(
+            warnings.warn(
                 f"Unknown setting {name!r} - the default value `{value!r}` will be used",
                 RuntimeWarning,
             )
@@ -657,12 +657,16 @@ def feature_flag(
         Raises:
             FeatureNotAllowedException: if raise_error is True and feature flag is disallowed
         """
-        # experimental is a top-level setting
-        if feature == EXPERIMENTAL:
-            allowed = self.get(EXPERIMENTAL) or False
-        # other feature flags are nested under feature flag
-        else:
-            allowed = self.get(f"{FEATURE_FLAG_PREFIX}.{feature}") or False
+        with warnings.catch_warnings():
+            warnings.filterwarnings("ignore", "Unknown setting", RuntimeWarning)
+
+            # experimental is a top-level setting
+            if feature == EXPERIMENTAL:
+                allowed = self.get(EXPERIMENTAL) or False
+            # other feature flags are nested under feature flag
+            else:
+                allowed = self.get(f"{FEATURE_FLAG_PREFIX}.{feature}") or False
+
         try:
             yield allowed
         finally:


Test patch:
Repository: meltano/meltano

Issue:
Enable image security scanning in CI
We can use (multiple) actions from the marketplace for this:
- https://github.com/marketplace/actions/setup-jfrog-cli
- https://github.com/marketplace/actions/container-image-scan
- https://github.com/marketplace/actions/anchore-container-scan

We will want to run these whenever new images are created, as well as whenever dependencies are updated. We'll also want to run this regularly (e.g. daily) as new security vulnerabilities to be discovered and reported despite us not changing anything.


Discussion:
Adding a comment in order to be pinged. 

Resolving patch:
diff --git a/.github/actions/docker-build-scan-push/check_sarif.py b/.github/actions/docker-build-scan-push/check_sarif.py
new file mode 100644
--- /dev/null
+++ b/.github/actions/docker-build-scan-push/check_sarif.py
@@ -0,0 +1,33 @@
+"""Check if the provided SARIF file has any violations at or above some severity level."""
+
+import argparse
+import json
+
+DEFAULT_SEVERITY_CUTOFF = 4.0
+
+parser = argparse.ArgumentParser()
+parser.add_argument(
+    "sarif_path",
+    help="The path to the SARIF file to be checked.",
+)
+parser.add_argument(
+    "--severity-cutoff",
+    help="Violations with a severity >= this value result in an exit code of 1"
+    + " - must be a number in the range [0.0, 10.0].",
+    type=float,
+    default=DEFAULT_SEVERITY_CUTOFF,
+)
+args = parser.parse_args()
+
+with open(args.sarif_path) as sarif_file:
+    sarif_data = json.load(sarif_file)
+
+first_run = sarif_data["runs"][0]
+triggered_rules = first_run["tool"]["driver"]["rules"]
+
+exit(  # noqa: WPS421
+    any(
+        float(rule["properties"]["security-severity"]) >= args.severity_cutoff
+        for rule in triggered_rules
+    )
+)


Test patch:
Repository: meltano/meltano

Issue:
meltano run should not require a command to execute a plugin
Per https://github.com/meltano/meltano/pull/6121#discussion_r895929389 it should be a simple change.

From @pandemicsyn the code is likely in https://github.com/meltano/meltano/blob/dfd9650170ff97fd69fdff7ac8dc7dc0841bf663/src/meltano/core/block/singer.py#L97 and we would need a docs update with this as well.


Discussion:
@aaronsteers I've added to Iteration 6 since this is a small change.
FYI @tayloramurphy I've got a little extra capacity this week so I'll plan on picking this up unless there's another more urgent unassigned issue that I can tackle in this iteration.
@cjohnhanson perfect thanks for the ping. Anything that's in the iteration but not assigned should be ready to be picked up by anyone!

Resolving patch:
diff --git a/src/meltano/cli/run.py b/src/meltano/cli/run.py
--- a/src/meltano/cli/run.py
+++ b/src/meltano/cli/run.py
@@ -65,7 +65,7 @@ async def run(
     Run a set of command blocks in series.
 
     Blocks are specified as a list of plugin names, e.g.
-    `meltano run some_extractor some_loader some_plugin:some_command` and are run in the order they are specified
+    `meltano run some_extractor some_loader some_plugin:some_optional_command` and are run in the order they are specified
     from left to right. A failure in any block will cause the entire run to abort.
 
     Multiple command blocks can be chained together or repeated, and tap/target pairs will automatically be linked:
diff --git a/src/meltano/core/block/singer.py b/src/meltano/core/block/singer.py
--- a/src/meltano/core/block/singer.py
+++ b/src/meltano/core/block/singer.py
@@ -94,9 +94,6 @@ async def start(self, *args, **kwargs):
         Raises:
             RunnerError: If the plugin process can not start.
         """
-        if self.command is None:
-            raise RunnerError("No command to run")
-
         try:
             self.process_handle = await self.invoker.invoke_async(
                 *args,


Test patch:
Repository: meltano/meltano

Issue:
bug: Meltano Upgrade on Windows stops with error "is not a valid Win32 application"
### Meltano Version

2.3.0

### Python Version

3.9

### Bug scope

CLI (options, error messages, logging, etc.)

### Operating System

Windows Server 2022 Standard

### Description

I have noticed on Windows if the file(symbolic link) `.meltano/run/bin` is present in your project when you run the upgrade via `meltano upgrade` or the `melatno ui` "Upgrade Button" you will get the following `WinError`.

`[WinError 193] %1 is not a valid Win32 application`

if you remove the file and run `meltano upgrade` the upgrade will complete. I am not sure what the long term ramification of deleting the file is.

![image](https://user-images.githubusercontent.com/55715401/180505118-6370a8ad-5a37-4dab-9216-aa539b1a3eae.png)


### Code

_No response_


Discussion:
I found running `meltano install`  will create a `.meltano/run/bin` in my test projects.    

Resolving patch:
diff --git a/src/meltano/core/project.py b/src/meltano/core/project.py
--- a/src/meltano/core/project.py
+++ b/src/meltano/core/project.py
@@ -123,13 +123,33 @@ def activate(cls, project: Project):
         Raises:
             OSError: if project cannot be activated due to unsupported OS
         """
+        import ctypes
+
         project.ensure_compatible()
 
         # create a symlink to our current binary
         try:
-            executable = Path(os.path.dirname(sys.executable), "meltano")
-            if executable.is_file():
-                project.run_dir().joinpath("bin").symlink_to(executable)
+            # check if running on Windows
+            if os.name == "nt":
+                executable = Path(sys.executable).parent / "meltano.exe"
+                # Admin privileges are required to create symlinks on Windows
+                if ctypes.windll.shell32.IsUserAnAdmin():
+                    if executable.is_file():
+                        project.run_dir().joinpath("bin").symlink_to(executable)
+                    else:
+                        logger.warn(
+                            "Could not create symlink: meltano.exe not "
+                            f"present in {str(Path(sys.executable).parent)}"
+                        )
+                else:
+                    logger.warn(
+                        "Failed to create symlink to 'meltano.exe': "
+                        "administrator privilege required"
+                    )
+            else:
+                executable = Path(sys.executable).parent / "meltano"
+                if executable.is_file():
+                    project.run_dir().joinpath("bin").symlink_to(executable)
         except FileExistsError:
             pass
         except OSError as error:


Test patch:
Repository: meltano/meltano

Issue:
Support Node v16
Currently building the Meltano UI with Node 16 results in a crash while building libsass. [That library is deprecated](https://sass-lang.com/blog/libsass-is-deprecated), so we should switch to using Dart-sass instead.

CC @alexmarple 


Discussion:

Resolving patch:
diff --git a/scripts/alembic_freeze.py b/scripts/alembic_freeze.py
--- a/scripts/alembic_freeze.py
+++ b/scripts/alembic_freeze.py
@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 
-"""Script to freeze the Meltano database - executed by the Makefile."""
+"""Script to freeze the Meltano database - executed by GitHub CI."""
 
 from __future__ import annotations
 


Test patch: