Repository: meltano/meltano

Issue:
How to catch dev dependencies that actually need to be included in the distributed presentations of Meltano?
Just got bit by this with `packaging` in https://github.com/meltano/meltano/pull/6276:

It's a transitive dependency required by `tox`, but `tox` is only a dev dependency. So, in CI tests, things look good because dev dependencies are installed by `poetry install`. But, when meltano is installed from an upstream source or by another tool like `pipx`, which is the case in end-to-end tests [here](https://gitlab.com/meltano/legacy-ci/meltano/-/jobs/2632994538#L63) (`pipx install ..`), then the transitive dev dependency `packaging` is not installed and errors are encountered: https://gitlab.com/meltano/legacy-ci/meltano/-/jobs/2632994539#L1237.

```console
$ meltano add orchestrator airflow
2022-06-23T19:10:30.061252Z [info     ] Environment 'dev' is active
No module named 'packaging'
```

I guess one easy way to ensure we don't fall in that scenario is to have tests that don't rely on Poetry to install Meltano.


Discussion:
Export the lockfile from from poetry to pin deps in a requirements.txt as part of the release process? 🤔 

> Export the lockfile from from poetry to pin deps in a requirements.txt as part of the release process? 🤔

@z3z1ma Yeah, I agree that `poetry export` is the way to go but it probably has to be done only during our testing stage, otherwise we'd have to do it with `poetry export --dev` for sdist and wheels, which is not ideal because that would include _all_ dev dependencies.

I particularly like the [`nox` + `poetry export --dev` approach](https://github.com/cjolowicz/cookiecutter-hypermodern-python/blob/6bcfea64890c148c9c834cc82820f433935b6701/%7B%7Bcookiecutter.project_name%7D%7D/noxfile.py#L160-L169) where the exported requirements.txt is used as a pip constraints file to install only dev dependencies specifically required by the dev task.
@pandemicsyn and @edgarrmondragon - What do you think of this being a part of 

- #6218?

At least when running in CI, we could use a pipx-installed version of Meltano instead of the Poetry-installed one.

> At least when running in CI, we could use a pipx-installed version of Meltano instead of the Poetry-installed one.

Probably a better discussion for #6218, but sounds fine  - at some point we'd probably want a test mirroring the contributor guide which might end up using poetry.
The SDK had this exact issue in the `v0.6.0` release so I went ahead and started a PR with `nox-poetry`, which runs tasks against the _installed_ form of the package using the wheel.

https://github.com/meltano/sdk/pull/780

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
new file mode 100644
--- /dev/null
+++ b/noxfile.py
@@ -0,0 +1,73 @@
+"""Nox configuration."""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+from textwrap import dedent
+
+try:
+    from nox_poetry import Session
+    from nox_poetry import session as nox_session
+except ImportError:
+    message = f"""\
+    Nox failed to import the 'nox-poetry' package.
+    Please install it using the following command:
+    {sys.executable} -m pip install nox-poetry"""
+    raise SystemExit(dedent(message)) from None
+
+
+package = "meltano"
+python_versions = ["3.10", "3.9", "3.8", "3.7"]
+main_python_version = "3.9"
+locations = "src", "tests", "noxfile.py"
+
+
+@nox_session(python=python_versions)
+def tests(session: Session) -> None:
+    """Execute pytest tests and compute coverage.
+
+    Args:
+        session: Nox session.
+    """
+    session.install(".")
+    session.install(
+        "coverage[toml]",
+        "freezegun",
+        "mock",
+        "pytest",
+        "pytest-asyncio",
+        "pytest-docker",
+        "requests-mock",
+    )
+
+    try:
+        session.run(
+            "coverage",
+            "run",
+            "--parallel",
+            "-m",
+            "pytest",
+            *session.posargs,
+            env={"NOX_CURRENT_SESSION": "tests"},
+        )
+    finally:
+        if session.interactive:
+            session.notify("coverage", posargs=[])
+
+
+@nox_session(python=main_python_version)
+def coverage(session: Session) -> None:
+    """Upload coverage data.
+
+    Args:
+        session: Nox session.
+    """
+    args = session.posargs or ["report"]
+
+    session.install("coverage[toml]")
+
+    if not session.posargs and any(Path().glob(".coverage.*")):
+        session.run("coverage", "combine")
+
+    session.run("coverage", *args)
diff --git a/src/meltano/core/upgrade_service.py b/src/meltano/core/upgrade_service.py
--- a/src/meltano/core/upgrade_service.py
+++ b/src/meltano/core/upgrade_service.py
@@ -66,20 +66,28 @@ def reload_ui(self):
             logging.error(f"Cannot restart from `{pid_file_path}`: {ex}")
 
     def _upgrade_package(self, pip_url: str | None, force: bool) -> bool:
+        fail_reason = None
+        instructions = ""
+
         meltano_file_path = "/src/meltano/__init__.py"
         editable = meltano.__file__.endswith(meltano_file_path)
         if editable and not force:
             meltano_dir = meltano.__file__[: -len(meltano_file_path)]
-            raise AutomaticPackageUpgradeError(
-                reason="it is installed from source",
-                instructions=f"navigate to `{meltano_dir}` and run `git pull`",
-            )
+            fail_reason = "it is installed from source"
+            instructions = f"navigate to `{meltano_dir}` and run `git pull`"
+
+        elif os.path.exists("/.dockerenv"):
+            fail_reason = "it is installed inside Docker"
+            instructions = "pull the latest Docker image using `docker pull meltano/meltano` and recreate any containers you may have created"
+
+        elif os.getenv("NOX_CURRENT_SESSION") == "tests":
+            fail_reason = "it is installed inside a Nox test session"
+            instructions = ""
 
-        in_docker = os.path.exists("/.dockerenv")
-        if in_docker:
+        if fail_reason:
             raise AutomaticPackageUpgradeError(
-                reason="it is installed inside Docker",
-                instructions="pull the latest Docker image using `docker pull meltano/meltano` and recreate any containers you may have created",
+                reason=fail_reason,
+                instructions=instructions,
             )
 
         pip_url = pip_url or "meltano"
@@ -112,7 +120,8 @@ def upgrade_package(self, pip_url: str | None = None, force: bool = False) -> bo
             click.echo(
                 f"{click.style('The `meltano` package could not be upgraded automatically', fg='red')} because {err.reason}."
             )
-            click.echo(f"To upgrade manually, {err.instructions}.")
+            if err.instructions:
+                click.echo(f"To upgrade manually, {err.instructions}.")
             return False
 
         click.echo("The `meltano` package has been upgraded.")


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -24,21 +24,32 @@ jobs:
         - { python-version: "3.8",  os: "ubuntu-latest",  backend-db: postgresql }
         - { python-version: "3.9",  os: "ubuntu-latest",  backend-db: postgresql }
         - { python-version: "3.10", os: "ubuntu-latest",  backend-db: postgresql }
-        - { python-version: "3.7",  os: "windows-latest", backend-db: sqlite } # We'd like to run Windows tests for all backend-dbs see https://github.com/meltano/meltano/issues/6281
-        - { python-version: "3.8",  os: "windows-latest", backend-db: sqlite }
-        - { python-version: "3.9",  os: "windows-latest", backend-db: sqlite }
-        - { python-version: "3.10", os: "windows-latest", backend-db: sqlite }
+        # We'd like to run Windows tests for all backend-dbs see https://github.com/meltano/meltano/issues/6281
+        # Windows tests for Python 3.7 and 3.10 are temporarily disabled - see https://github.com/meltano/meltano/issues/6479
+        # - { python-version: "3.7",  os: "windows-2022",   backend-db: sqlite }
+        - { python-version: "3.8",  os: "windows-2022",   backend-db: sqlite }
+        - { python-version: "3.9",  os: "windows-2022",   backend-db: sqlite }
+        # - { python-version: "3.10", os: "windows-2022",   backend-db: sqlite }
       fail-fast: false
 
     name: "Pytest on py${{ matrix.python-version }} (OS: ${{ matrix.os }}, DB: ${{ matrix.backend-db }})"
     runs-on: ${{ matrix.os }}
     env:
       PYTEST_MARKERS: not concurrent
+      FORCE_COLOR: 1
 
     steps:
     - name: Check out the repository
       uses: actions/checkout@v3.0.2
 
+    - name: Setup Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4.1.0
+      with:
+        python-version: ${{ matrix.python-version }}
+        architecture: x64
+        cache: 'pip'
+        cache-dependency-path: 'poetry.lock'
+
     - name: Install Poetry
       env:
         PIP_CONSTRAINT: .github/workflows/resources/constraints.txt
@@ -46,13 +57,6 @@ jobs:
         pipx install poetry
         poetry --version
 
-    - name: Setup Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4.1.0
-      with:
-        python-version: ${{ matrix.python-version }}
-        architecture: x64
-        cache: 'poetry'
-
     - name: Upgrade pip
       env:
         PIP_CONSTRAINT: .github/workflows/resources/constraints.txt
@@ -60,10 +64,10 @@ jobs:
         pip install pip
         pip --version
 
-    - name: Install Dependencies
+    - name: Configure Poetry
       run: |
         poetry env use "${{ matrix.python-version }}"
-        poetry install
+        poetry debug
 
     - name: Start Postgres Container
       if: always() && (matrix.backend-db == 'postgresql')
@@ -82,6 +86,13 @@ jobs:
       run: |
         docker ps -a
 
+    - name: Install Nox
+      env:
+        PIP_CONSTRAINT: .github/workflows/resources/constraints.txt
+      run: |
+        pipx install nox
+        pipx inject nox nox-poetry
+
     - name: Run pytest
       env:
         PYTEST_BACKEND: ${{ matrix.backend-db }}
@@ -94,7 +105,7 @@ jobs:
         POSTGRES_PASSWORD: postgres
         POSTGRES_DB: pytest_warehouse
       run: |
-        poetry run coverage run --parallel -m pytest -m "${{ env.PYTEST_MARKERS }}"
+        nox -rs tests --python=${{ matrix.python-version }} -- -m "${{ env.PYTEST_MARKERS }}"
 
     - name: Upload coverage data
       if: always() && (matrix.python-version == '3.9')