Repository: matplotlib/matplotlib
Pull request: Simplify appveyor to only use conda
Closes #24394

## PR Summary

Avoid mixing conda and pip (except for those pip dependencies in the environment.yml)
Should fix appveyor tests failing due to pip pywin32 version.

## PR Checklist

<!-- Please mark any checkboxes that do not apply to this PR as [N/A]. -->
**Tests and Styling**
- [x] Has pytest style unit tests (and `pytest` passes).
- [x] Is [Flake 8](https://flake8.pycqa.org/en/latest/) compliant (install `flake8-docstrings` and run `flake8 --docstring-convention=all`).

**Documentation**
- [x] Documentation is sphinx and numpydoc compliant (the docs should [build](https://matplotlib.org/devel/documenting_mpl.html#building-the-docs) without error).
- [x] New plotting related features are documented with examples.

**Release Notes**
- [x] New features are marked with a `.. versionadded::` directive in the docstring and documented in `doc/users/next_whats_new/`
- [x] API changes are marked with a `.. versionchanged::` directive in the docstring and documented in `doc/api/next_api_changes/`
- [x] Release notes conform with instructions in  `next_whats_new/README.rst` or `next_api_changes/README.rst`

<!--
Thank you so much for your PR!  To help us review your contribution, please
consider the following points:

- A development guide is available at https://matplotlib.org/devdocs/devel/index.html.

- Help with git and github is available at
  https://matplotlib.org/devel/gitwash/development_workflow.html.

- Create a separate branch for your changes and open the PR from this branch. Please avoid working on `main`.

- The PR title should summarize the changes, for example "Raise ValueError on
  non-numeric input to set_xlim".  Avoid non-descriptive titles such as
  "Addresses issue #8576".

- The summary should provide at least 1-2 sentences describing the pull request
  in detail (Why is this change required?  What problem does it solve?) and
  link to any relevant issues.

- If you are contributing fixes to docstrings, please pay attention to
  http://matplotlib.org/devel/documenting_mpl.html#formatting.  In particular,
  note the difference between using single backquotes, double backquotes, and
  asterisks in the markup.

We understand that PRs can sometimes be overwhelming, especially as the
reviews start coming in.  Please let us know if the reviews are unclear or
the recommended next step seems overly demanding, if you would like help in
addressing a reviewer's comments, or if you have been waiting too long to hear
back on your PR.
-->


Problem statement:
[TST]: Appveyor Qt tests failing
### Summary

Appveyor Qt sigint tests appear to be consistently failing due DLL loading

<details>
<summary> Relevant log portion</summary>


```
__________________________ test_sigint[show-kwargs0] __________________________
[gw0] win32 -- Python 3.8.13 C:\Miniconda3-x64\envs\test-environment\python.exe
target = 'show', kwargs = {'block': True}
    @pytest.mark.backend('QtAgg', skip_on_importerror=True)
    @pytest.mark.parametrize("target, kwargs", [
        ('show', {'block': True}),
        ('pause', {'interval': 10})
    ])
    def test_sigint(target, kwargs):
        backend = plt.get_backend()
        proc = WaitForStringPopen(
            [sys.executable, "-c",
             inspect.getsource(_test_sigint_impl) +
             f"\n_test_sigint_impl({backend!r}, {target!r}, {kwargs!r})"])
        try:
            proc.wait_for('DRAW')
>           stdout, _ = proc.communicate(timeout=_test_timeout)
lib\matplotlib\tests\test_backend_qt.py:130: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda3-x64\envs\test-environment\lib\subprocess.py:1028: in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <matplotlib.tests.test_backend_qt.WaitForStringPopen object at 0x0000004E17E65CD0>
input = None, endtime = 526.437, orig_timeout = 60
    def _communicate(self, input, endtime, orig_timeout):
        # Start reader threads feeding into a list hanging off of this
        # object, unless they've already been started.
        if self.stdout and not hasattr(self, "_stdout_buff"):
            self._stdout_buff = []
            self.stdout_thread = \
                    threading.Thread(target=self._readerthread,
                                     args=(self.stdout, self._stdout_buff))
            self.stdout_thread.daemon = True
            self.stdout_thread.start()
        if self.stderr and not hasattr(self, "_stderr_buff"):
            self._stderr_buff = []
            self.stderr_thread = \
                    threading.Thread(target=self._readerthread,
                                     args=(self.stderr, self._stderr_buff))
            self.stderr_thread.daemon = True
            self.stderr_thread.start()
    
        if self.stdin:
            self._stdin_write(input)
    
        # Wait for the reader threads, or time out.  If we time out, the
        # threads remain reading and the fds left open in case the user
        # calls communicate again.
        if self.stdout is not None:
            self.stdout_thread.join(self._remaining_time(endtime))
            if self.stdout_thread.is_alive():
>               raise TimeoutExpired(self.args, orig_timeout)
E               subprocess.TimeoutExpired: Command '['C:\\Miniconda3-x64\\envs\\test-environment\\python.exe', '-c', "def _test_sigint_impl(backend, target_name, kwargs):\n    import sys\n    import matplotlib.pyplot as plt\n    import os\n    import threading\n\n    plt.switch_backend(backend)\n    from matplotlib.backends.qt_compat import QtCore  # noqa\n\n    def interrupter():\n        if sys.platform == 'win32':\n            import win32api\n            win32api.GenerateConsoleCtrlEvent(0, 0)\n        else:\n            import signal\n            os.kill(os.getpid(), signal.SIGINT)\n\n    target = getattr(plt, target_name)\n    timer = threading.Timer(1, interrupter)\n    fig = plt.figure()\n    fig.canvas.mpl_connect(\n        'draw_event',\n        lambda *args: print('DRAW', flush=True)\n    )\n    fig.canvas.mpl_connect(\n        'draw_event',\n        lambda *args: timer.start()\n    )\n    try:\n        target(**kwargs)\n    except KeyboardInterrupt:\n        print('SUCCESS', flush=True)\n\n_test_sigint_impl('QtAgg', 'show', {'block': True})"]' timed out after 60 seconds
C:\Miniconda3-x64\envs\test-environment\lib\subprocess.py:1401: TimeoutExpired
----------------------------- Captured log setup ------------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend QtAgg version 5.15.7.
---------------------------- Captured stderr call -----------------------------
Windows fatal exception: code 0xc0000139
Current thread 0x00000900 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1166 in create_module
  File "<frozen importlib._bootstrap>", line 556 in module_from_spec
  File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<string>", line 12 in interrupter
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254 in run
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932 in _bootstrap_inner
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 890 in _bootstrap
Thread 0x00000344 (most recent call first):
  File "c:\projects\matplotlib\lib\matplotlib\backends\qt_compat.py", line 168 in _exec
  File "c:\projects\matplotlib\lib\matplotlib\backends\backend_qt.py", line 1022 in mainloop
  File "c:\projects\matplotlib\lib\matplotlib\backend_bases.py", line 3554 in show
  File "c:\projects\matplotlib\lib\matplotlib\pyplot.py", line 421 in show
  File "<string>", line 30 in _test_sigint_impl
  File "<string>", line 34 in <module>
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254, in run
    self.function(*self.args, **self.kwargs)
  File "<string>", line 12, in interrupter
ImportError: DLL load failed while importing win32api: The specified procedure could not be found.
---------------------------- Captured log teardown ----------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend agg version v2.2.
_________________________ test_sigint[pause-kwargs1] __________________________
[gw0] win32 -- Python 3.8.13 C:\Miniconda3-x64\envs\test-environment\python.exe
target = 'pause', kwargs = {'interval': 10}
    @pytest.mark.backend('QtAgg', skip_on_importerror=True)
    @pytest.mark.parametrize("target, kwargs", [
        ('show', {'block': True}),
        ('pause', {'interval': 10})
    ])
    def test_sigint(target, kwargs):
        backend = plt.get_backend()
        proc = WaitForStringPopen(
            [sys.executable, "-c",
             inspect.getsource(_test_sigint_impl) +
             f"\n_test_sigint_impl({backend!r}, {target!r}, {kwargs!r})"])
        try:
            proc.wait_for('DRAW')
            stdout, _ = proc.communicate(timeout=_test_timeout)
        except:
            proc.kill()
            stdout, _ = proc.communicate()
            raise
        print(stdout)
>       assert 'SUCCESS' in stdout
E       AssertionError: assert 'SUCCESS' in '\n'
lib\matplotlib\tests\test_backend_qt.py:136: AssertionError
----------------------------- Captured log setup ------------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend QtAgg version 5.15.7.
---------------------------- Captured stdout call -----------------------------
---------------------------- Captured stderr call -----------------------------
Windows fatal exception: code 0xc0000139
Current thread 0x00000664 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1166 in create_module
  File "<frozen importlib._bootstrap>", line 556 in module_from_spec
  File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<string>", line 12 in interrupter
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254 in run
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932 in _bootstrap_inner
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 890 in _bootstrap
Thread 0x00000538 (most recent call first):
  File "c:\projects\matplotlib\lib\matplotlib\backends\qt_compat.py", line 168 in _exec
  File "c:\projects\matplotlib\lib\matplotlib\backends\backend_qt.py", line 408 in start_event_loop
  File "c:\projects\matplotlib\lib\matplotlib\pyplot.py", line 557 in pause
  File "<string>", line 30 in _test_sigint_impl
  File "<string>", line 34 in <module>
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254, in run
    self.function(*self.args, **self.kwargs)
  File "<string>", line 12, in interrupter
ImportError: DLL load failed while importing win32api: The specified procedure could not be found.
---------------------------- Captured log teardown ----------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend agg version v2.2.
```

</details>

### Proposed fix

_No response_

Hints from issue discussion:
Full log: https://ci.appveyor.com/project/matplotlib/matplotlib/builds/45303213/job/p0makskomsa3rssy
Neither Qt or PyQt had releases on conda-forge in the last few days which was my first guess at what is going wrong.

Sorry I miss-read these logs on #24380 .
https://www.appveyor.com/updates/2022/11/07/

Probably related to the appveyor platform updates that rolled out today
Possibly also related:

https://github.com/matplotlib/matplotlib/blob/9f96205464930c9064eb20af462c79ac47c6b254/.appveyor.yml#L57-L59

Could be that the issue wasn't "pip vs conda", but rather some other factor... not sure what, though
https://ci.appveyor.com/project/ksunden/appveyor-test/build/job/akqg7r4vfncf862d

Replicated with a stripped down version of the tests (didn't install mpl, but installed all requirements still)
Okay, I know what is happening now:

Pywin32 [released yesterday](https://pypi.org/project/pywin32/#history) and so is on version 305 on pypi, but version 304 on conda forge

We attempt to install via conda because of the reasons indicated by the comment above. However, since the pypi version is higher, the conda install gets overridden with the PyPI version since pywin32 is specified in `requirements/testing/extra.txt` and it is installed with `--upgrade`

The pip version is broken for the conda install.

Our options are:
- wait for conda forge to get 305, and just ignore it (maybe add some more comments)
- remove pywin32 from requirements/testing/extra.txt (and special case it for azure pipelines if we care to include it there... it is already special cased for appveyor, and only applies to Windows CI, so not github actions)
- See if reordering the conda install fixes the problem (testing this now on my test repo) (not a huge fan of doing conda package installs _after_ pip package installs, but if it works...)
Reordering works with the caveat that it still shows up as the pypi version in `conda list`, and so is likely to be confusing if anybody is looking at a similar problem in the future...
The mixing of conda and pip on CI is not great.  Is it plausible to go pure conda on appveyor?

Another option is "does appveyor actually add coverage we do not get from azure + gha?" If not we should just drop the whole CI platform....
We do get windows from azure (GHA is not currently configured for windows, could be added, but haven't looked at usage caps/simultaneous run caps or anything).

We do not have any other tests which use conda, if that is important to us. (in particular, conda can be a little slow when it comes to updating qt dependencies... I know over at pyqtgraph we made explicitly sure to work with the version of qt available via conda, and CI as much)

As for going full conda, no reason why it couldn't be done, just would be an additional place to update deps as it can't use the requirements txt files since package names are sometimes different.
We have an environment.yml. I think that could be used for appveyor.

Commit message: Just skip memleak test on windows entirely

Patch to review:
diff --git a/.appveyor.yml b/.appveyor.yml
index df7536f16c7e..0ed198d8f359 100644
--- a/.appveyor.yml
+++ b/.appveyor.yml
@@ -18,17 +18,14 @@ environment:
     PYTHONIOENCODING: UTF-8
     PYTEST_ARGS: -raR --numprocesses=auto --timeout=300 --durations=25
                  --cov-report= --cov=lib --log-level=DEBUG
-    PINNEDVERS: "pyzmq!=21.0.0,!=22.0.0"
 
   matrix:
     - PYTHON_VERSION: "3.8"
       CONDA_INSTALL_LOCN: "C:\\Miniconda3-x64"
       TEST_ALL: "no"
-      EXTRAREQS: "-r requirements/testing/extra.txt"
     - PYTHON_VERSION: "3.9"
       CONDA_INSTALL_LOCN: "C:\\Miniconda3-x64"
       TEST_ALL: "no"
-      EXTRAREQS: "-r requirements/testing/extra.txt"
 
 # We always use a 64-bit machine, but can build x86 distributions
 # with the PYTHON_ARCH variable
@@ -52,19 +49,13 @@ install:
   - conda config --prepend channels conda-forge
 
   # For building, use a new environment
-  - conda create -q -n test-environment python=%PYTHON_VERSION% tk "pip<22.0"
-  - activate test-environment
-  # pull pywin32 from conda because on py38 there is something wrong with finding
-  # the dlls when installed from pip
+  # Add python version to environment
+  # `^ ` escapes spaces for indentation
+  - echo ^ ^ - python=%PYTHON_VERSION% >> environment.yml
+  - conda env create -f environment.yml
+  - activate mpl-dev
   - conda install -c conda-forge pywin32
-  # install pyqt from conda-forge
-  - conda install -c conda-forge pyqt
   - echo %PYTHON_VERSION% %TARGET_ARCH%
-  # Install dependencies from PyPI.
-  - python -m pip install --upgrade -r requirements/testing/all.txt %EXTRAREQS% %PINNEDVERS%
-  # Install optional dependencies from PyPI.
-  # Sphinx is needed to run sphinxext tests
-  - python -m pip install --upgrade sphinx
   # Show the installed packages + versions
   - conda list
 
@@ -104,7 +95,7 @@ artifacts:
     type: zip
 
 on_finish:
-  - pip install codecov
+  - conda install codecov
   - codecov -e PYTHON_VERSION PLATFORM
 
 on_failure:
diff --git a/environment.yml b/environment.yml
index c9b7aa610720..e504b4b5e10b 100644
--- a/environment.yml
+++ b/environment.yml
@@ -44,7 +44,7 @@ dependencies:
   - coverage
   - flake8>=3.8
   - flake8-docstrings>=1.4.0
-  - gtk3
+  - gtk4
   - ipykernel
   - nbconvert[execute]!=6.0.0,!=6.0.1
   - nbformat!=5.0.0,!=5.0.1
diff --git a/lib/matplotlib/tests/test_backends_interactive.py b/lib/matplotlib/tests/test_backends_interactive.py
index e5e6b295e798..d22bd6969f91 100644
--- a/lib/matplotlib/tests/test_backends_interactive.py
+++ b/lib/matplotlib/tests/test_backends_interactive.py
@@ -166,6 +166,8 @@ def test_interactive_backend(env, toolbar):
     if env["MPLBACKEND"] == "macosx":
         if toolbar == "toolmanager":
             pytest.skip("toolmanager is not implemented for macosx.")
+    if env["MPLBACKEND"] == "wx":
+        pytest.skip("wx backend is deprecated; tests failed on appveyor")
     proc = _run_helper(_test_interactive_impl,
                        json.dumps({"toolbar": toolbar}),
                        timeout=_test_timeout,
@@ -560,6 +562,8 @@ def _test_figure_leak():
 
 
 # TODO: "0.1" memory threshold could be reduced 10x by fixing tkagg
+@pytest.mark.skipif(sys.platform == "win32",
+                    reason="appveyor tests fail; gh-22988 suggests reworking")
 @pytest.mark.parametrize("env", _get_testable_interactive_backends())
 @pytest.mark.parametrize("time_mem", [(0.0, 2_000_000), (0.1, 30_000_000)])
 def test_figure_leak_20490(env, time_mem):
@@ -568,6 +572,9 @@ def test_figure_leak_20490(env, time_mem):
     # We haven't yet directly identified the leaks so test with a memory growth
     # threshold.
     pause_time, acceptable_memory_leakage = time_mem
+    if env["MPLBACKEND"] == "wx":
+        pytest.skip("wx backend is deprecated; tests failed on appveyor")
+
     if env["MPLBACKEND"] == "macosx" or (
             env["MPLBACKEND"] == "tkagg" and sys.platform == 'darwin'
     ):
diff --git a/lib/matplotlib/tests/test_font_manager.py b/lib/matplotlib/tests/test_font_manager.py
index b1b5b37f039a..8aaeb863ece3 100644
--- a/lib/matplotlib/tests/test_font_manager.py
+++ b/lib/matplotlib/tests/test_font_manager.py
@@ -73,7 +73,8 @@ def test_otf():
             assert res == is_opentype_cff_font(f.fname)
 
 
-@pytest.mark.skipif(not has_fclist, reason='no fontconfig installed')
+@pytest.mark.skipif(sys.platform == "win32" or not has_fclist,
+                    reason='no fontconfig installed')
 def test_get_fontconfig_fonts():
     assert len(_get_fontconfig_fonts()) > 1
 
diff --git a/lib/matplotlib/tests/test_getattr.py b/lib/matplotlib/tests/test_getattr.py
index 71556b5b4c71..76ae3d7143ed 100644
--- a/lib/matplotlib/tests/test_getattr.py
+++ b/lib/matplotlib/tests/test_getattr.py
@@ -4,14 +4,20 @@
 import matplotlib
 import pytest
 
-# Get the names of all matplotlib submodules, except for the unit tests.
-module_names = [m.name for m in walk_packages(path=matplotlib.__path__,
-                                              prefix=f'{matplotlib.__name__}.')
-                if not m.name.startswith(__package__)]
+# Get the names of all matplotlib submodules,
+# except for the unit tests and private modules.
+module_names = [
+    m.name
+    for m in walk_packages(
+        path=matplotlib.__path__, prefix=f"{matplotlib.__name__}."
+    )
+    if not m.name.startswith(__package__)
+    and not any(x.startswith("_") for x in m.name.split("."))
+]
 
 
-@pytest.mark.parametrize('module_name', module_names)
-@pytest.mark.filterwarnings('ignore::DeprecationWarning')
+@pytest.mark.parametrize("module_name", module_names)
+@pytest.mark.filterwarnings("ignore::DeprecationWarning")
 def test_getattr(module_name):
     """
     Test that __getattr__ methods raise AttributeError for unknown keys.
@@ -21,8 +27,8 @@ def test_getattr(module_name):
         module = import_module(module_name)
     except (ImportError, RuntimeError) as e:
         # Skip modules that cannot be imported due to missing dependencies
-        pytest.skip(f'Cannot import {module_name} due to {e}')
+        pytest.skip(f"Cannot import {module_name} due to {e}")
 
-    key = 'THIS_SYMBOL_SHOULD_NOT_EXIST'
+    key = "THIS_SYMBOL_SHOULD_NOT_EXIST"
     if hasattr(module, key):
         delattr(module, key)


Code review comments:
diff_hunk: @@ -4,14 +4,20 @@
 import matplotlib
 import pytest
 
-# Get the names of all matplotlib submodules, except for the unit tests.
-module_names = [m.name for m in walk_packages(path=matplotlib.__path__,
-                                              prefix=f'{matplotlib.__name__}.')
-                if not m.name.startswith(__package__)]
+# Get the names of all matplotlib submodules,
+# except for the unit tests and private modules.
+module_names = [
+    m.name
+    for m in walk_packages(
+        path=matplotlib.__path__, prefix=f"{matplotlib.__name__}."
+    )
+    if not m.name.startswith(__package__)
+    and not any(x.startswith("_") for x in m.name.split("."))
+]
 
 
-@pytest.mark.parametrize('module_name', module_names)
-@pytest.mark.filterwarnings('ignore::DeprecationWarning')
+@pytest.mark.parametrize("module_name", module_names)
path: lib/matplotlib/tests/test_getattr.py
text: @user1:
Is this and the following from a code formatter? My understanding is that we don't want code churn like this. If we want to introduce more strict formatting, we need to decide this centrally and introduce this in a coordinated way.

@author:
Guilty... I did apply `black` (only to this file) when flake8 started complaining about the formatting of the module name filter

I can revert the quote changes if we want.