Repository: jupyterlab/pytest-check-links (language: python)

Issue:
Fixed deprication warning (#13)

Modernize dependencies, squash warnings?
While working on #11, #12, I've found some things annoying.

The following warnings end up in end users test logs:

`plugin.py:168: DeprecationWarning: This method will be removed in future versions.  Use 'tree.iter()' or 'list(tree.iter())' instead.`

- just what it says?

`PytestDeprecationWarning: direct construction of CheckLinks has been deprecated, please use CheckLinks.from_parent`

  - require whatever version introduces `from_parent`

Less pressing, but still annoying are our test cases:

`PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time`
  - don't use `copy_example`... just `shutil.copy` into testdir?

Finally, keeping 3.5 and 2.7 happy is annoying
  - perhaps drop them?
  - ...and start testing on `3.9-nightly`?
     - the `MutableMapping`pocalypse hits `html5lib`, `requests-cache`, etc. however
  - probably can't do both at once
Fixed deprication warning (#13)



Discussion / hints:

Given that 2.7 is past EOL, I say we drop it.  Python 3.5 is still supported in JupyterLab, so I say we keep it for now.  The others I don't see as blockers for release.
I don't think any of them are release blockers... Just stuff that bothered
me while down in the guts of the thing, and to create a little SEO juice
for the actual error message texts.

One thing I forgot to mention would be the ability to use lxml if
available, since html5lib can use it transparently... Though the parsing is
rather naive at present, and should probably be lru cached, at least,
before doing _yet more_ dependency stuff.

On Wed, Apr 29, 2020, 16:53 Steven Silvester <notifications@github.com>
wrote:

> Given that 2.7 is past EOL, I say we drop it. Python 3.5 is still
> supported in JupyterLab, so I say we keep it for now. The others I don't
> see as blockers for release.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/jupyterlab/pytest-check-links/issues/13#issuecomment-621457929>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAALCREAEVNDICKLW4SJXQLRPCHUFANCNFSM4MT53L7Q>
> .
>

Hello, this project looks really useful, should I try to fix the deprication errors? 
@AnnaDinaburgVulikh That would be marvelous!

I think folk have thus far been kinda willing to deal with it, since it's "just" docs, but clean runs benefit everyone!


Fix patch:
diff --git a/README.md b/README.md
index 119cb10..8a4b833 100644
--- a/README.md
+++ b/README.md
@@ -3,8 +3,10 @@
 pytest plugin that checks URLs for HTML-containing files.
 
 [![codecov](https://codecov.io/gh/jupyterlab/pytest-check-links/branch/master/graph/badge.svg)](https://codecov.io/gh/jupyterlab/pytest-check-links)
-[![Build Status](https://travis-ci.com/jupyterlab/pytest-check-links.svg?branch=master)](https://travis-ci.com/jupyterlab/pytest-check-links)
+[![Tests](https://github.com/jupyterlab/pytest-check-links/workflows/Tests/badge.svg)](https://github.com/jupyterlab/pytest-check-links/actions?query=workflow%3ATests+branch%3Amaster)
 [![PyPI version](https://badge.fury.io/py/pytest-check-links.svg)](https://badge.fury.io/py/pytest-check-links)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pytest-check-links)
+
 
 ## Supported files
 
diff --git a/examples/markdown.md b/examples/markdown.md
index 770debf..210df16 100644
--- a/examples/markdown.md
+++ b/examples/markdown.md
@@ -24,4 +24,4 @@ Some text to show that the reference links can follow later.
 
 [arbitrary case-insensitive reference text]: https://www.mozilla.org
 [1]: http://slashdot.org
-[link text itself]: http://www.reddit.com
+[link text itself]: https://www.w3.org
diff --git a/examples/rst.rst b/examples/rst.rst
index 8048c4f..e44a857 100644
--- a/examples/rst.rst
+++ b/examples/rst.rst
@@ -15,7 +15,7 @@ reStructuredText file
 definitions <http://slashdot.org>`__
 
 Or leave it empty and use the `link text
-itself <http://www.reddit.com>`__.
+itself <https://www.w3.org>`__.
 
 URLs will automatically get turned into
 links. http://www.example.com or http://www.example.com/404.
diff --git a/examples/setup.cfg b/examples/setup.cfg
new file mode 100644
index 0000000..4c69bb5
--- /dev/null
+++ b/examples/setup.cfg
@@ -0,0 +1,4 @@
+[tool:pytest]
+filterwarnings =
+    ignore::pytest.PytestAssertRewriteWarning
+    ignore:the imp module is deprecated in favour of importlib.*:DeprecationWarning
diff --git a/setup.cfg b/setup.cfg
index 85fcc97..7cecff3 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -7,17 +7,28 @@ description-file = README.md
 long-description-content-type = text/markdown
 home-page = https://github.com/jupyterlab/pytest-check-links
 license = BSD-3-Clause
-python-requires = >=3.5
+python-requires = >=3.6
 classifier =
+    Framework :: Jupyter
+    Framework :: Pytest
     Intended Audience :: Developers
     License :: OSI Approved :: BSD License
     Operating System :: OS Independent
     Programming Language :: Python
-    Programming Language :: Python :: 3
+    Programming Language :: Python :: 3 :: Only
+    Programming Language :: Python :: 3.6
+    Programming Language :: Python :: 3.7
+    Programming Language :: Python :: 3.8
+    Programming Language :: Python :: 3.9
+    Topic :: Documentation
+    Topic :: Documentation :: Sphinx
+    Topic :: Software Development :: Quality Assurance
     Topic :: Software Development :: Testing
 keywords =
-    setup
-    distutils
+    testing
+    documentation
+    links
+    html
 
 [files]
 packages =
@@ -38,3 +49,10 @@ ignore=E128
 
 [flake8]
 ignore=E128
+
+[tool:pytest]
+pytester_example_dir = examples
+
+filterwarnings =
+    ignore::FutureWarning:
+    ignore::DeprecationWarning:requests_cache


Test patch:
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 1ff6000..d3ff428 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -13,22 +13,14 @@ jobs:
   linux:
     name: ${{ matrix.PLATFORM }} py${{ matrix.PYTHON_VERSION }}
     runs-on: ${{ matrix.PLATFORM }}
-    env:
-      CI: True
-      PYTHON_VERSION: ${{ matrix.PYTHON_VERSION }}
     strategy:
       fail-fast: false
       matrix:
-        PYTHON_VERSION: ['3.6', '3.8']
         PLATFORM: ['ubuntu-latest', 'macos-latest',  'windows-latest']
+        PYTHON_VERSION: ['3.6', '3.9']
     steps:
       - name: Checkout
         uses: actions/checkout@v1
-      - name: Cache pip
-        uses: actions/cache@v1
-        with:
-          path: ~/.cache/pip
-          key: ${{ runner.os }}-py-${{ matrix.PYTHON_VERSION }}-pip-${{ hashFiles('**/setup.py') }}
       - name: Setup python ${{ matrix.PYTHON_VERSION }}
         uses: actions/setup-python@v1
         with:
@@ -36,8 +28,22 @@ jobs:
           architecture: 'x64'
       - name: Install python dependencies
         run: |
-          pip install setuptools pip --upgrade --user
-          pip install --upgrade .[cache] pytest-cov codecov
+          pip install --upgrade --user setuptools pip wheel
+      - name: Get pip cache dir
+        id: pip-cache
+        run: |
+          echo "::set-output name=dir::$(pip cache dir)"
+      - name: Cache pip
+        uses: actions/cache@v1
+        with:
+          path: ${{ steps.pip-cache.outputs.dir }}
+          key: ${{ runner.os }}-pip-${{ matrix.PYTHON_VERSION }}-${{ hashFiles('requirements.txt', 'setup.cfg') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-${{ matrix.PYTHON_VERSION }}-
+            ${{ runner.os }}-pip-
+      - name: Install package and dependencies
+        run: |
+          pip install --upgrade -vv .[cache] pytest-cov codecov
       - name: Show python environment
         run: |
           python --version
@@ -45,7 +51,7 @@ jobs:
       - name: Run python tests
         # disable automatic inclusion for coverage
         run: |
-          pytest -p no:check-links --cov pytest_check_links --cov-report term-missing
+          pytest -vv -p no:check-links --cov pytest_check_links --cov-report term-missing
       - name: Upload coverage
         run: |
           codecov
diff --git a/pytest.ini b/pytest.ini
deleted file mode 100644
index a4aeb61..0000000
--- a/pytest.ini
+++ /dev/null
@@ -1,2 +0,0 @@
-[pytest]
-pytester_example_dir = examples
diff --git a/pytest_check_links/__main__.py b/pytest_check_links/__main__.py
index 0c74f69..8e2d738 100644
--- a/pytest_check_links/__main__.py
+++ b/pytest_check_links/__main__.py
@@ -6,6 +6,8 @@
 these plugins with the py.test command line option
 "-p no:<plugin-name-here>".
 """
+# pragma: no cover
+
 import sys
 
 
diff --git a/pytest_check_links/plugin.py b/pytest_check_links/plugin.py
index 0bef8cc..d45137f 100644
--- a/pytest_check_links/plugin.py
+++ b/pytest_check_links/plugin.py
@@ -167,7 +167,7 @@ def links_in_html(base_name, parent, html):
     """
     parsed = html5lib.parse(html, namespaceHTMLElements=False)
 
-    for element in parsed.getiterator():
+    for element in parsed.iter():
         url = None
         tag = element.tag
         if tag == 'a':
diff --git a/test/__init__.py b/test/__init__.py
index 2f8598e..e69de29 100644
--- a/test/__init__.py
+++ b/test/__init__.py
@@ -1,4 +0,0 @@
-import os
-
-here = os.path.dirname(os.path.abspath(__file__))
-examples = os.path.join(os.path.dirname(here), 'examples')
diff --git a/test/conftest.py b/test/conftest.py
index c6481d5..180af97 100644
--- a/test/conftest.py
+++ b/test/conftest.py
@@ -1,1 +1,20 @@
+import sys
+import pytest
+
 pytest_plugins = ["pytester"]
+
+skip_pywin32 = pytest.mark.skipif(sys.platform == "win32", reason="pywin32 double import")
+
+
+@pytest.fixture
+def anchor_args():
+    return ["-v", "--check-links", "--check-anchors"]
+
+@pytest.fixture
+def base_args():
+    return ["-v", "--check-links", "--check-links-cache"]
+
+
+@pytest.fixture
+def memory_args(base_args):
+    return base_args + ["--check-links-cache-backend", "memory"]
diff --git a/test/test_anchors.py b/test/test_anchors.py
index 9b3c4fc..820f42e 100644
--- a/test/test_anchors.py
+++ b/test/test_anchors.py
@@ -1,13 +1,6 @@
-from . import examples
-
 import pytest
 
 
-@pytest.fixture
-def anchor_args():
-    return ["-v", "--check-links", "--check-anchors"]
-
-
 def test_anchors_local_self(testdir, anchor_args):
     testdir.copy_example('anchors_self.html')
     result = testdir.runpytest(*anchor_args)
diff --git a/test/test_cache.py b/test/test_cache.py
index 6fe3f26..8ffc1cd 100644
--- a/test/test_cache.py
+++ b/test/test_cache.py
@@ -8,18 +8,6 @@
 
 import requests_cache
 
-from . import examples
-
-
-@pytest.fixture
-def base_args():
-    return ["-v", "--check-links", "--check-links-cache"]
-
-
-@pytest.fixture
-def memory_args(base_args):
-    return base_args + ["--check-links-cache-backend", "memory"]
-
 
 def assert_sqlite(testdir, name=None, tmpdir=None, exists=True):
     name = name or ".pytest-check-links-cache.sqlite"
@@ -59,7 +47,10 @@ def test_cache_expiry(testdir, base_args, cache_name, tmpdir):
     t3 = time.time()
     result.assert_outcomes(**expected)
 
-    assert t1 - t0 > t3 - t2, "cache did not make second run faster"
+    d0 = t1 - t0
+    d1 = t3 - t2
+
+    assert d0 > d1, "cache did not make second run faster"
 
     time.sleep(2)
 
@@ -68,7 +59,10 @@ def test_cache_expiry(testdir, base_args, cache_name, tmpdir):
     t5 = time.time()
     result.assert_outcomes(**expected)
 
-    assert t5 - t4 > t3 - t2, "cache did not expire"
+    d2 = t5 - t4
+    d3 = t3 - t2
+
+    assert d2 > d3, "cache did not expire"
 
 
 def test_cache_memory(testdir, memory_args):
diff --git a/test/test_check_links.py b/test/test_check_links.py
index f8190f7..7c6938d 100644
--- a/test/test_check_links.py
+++ b/test/test_check_links.py
@@ -1,4 +1,4 @@
-from . import examples
+from .conftest import skip_pywin32
 
 
 def test_ipynb(testdir):
@@ -11,6 +11,7 @@ def test_markdown(testdir):
     result = testdir.runpytest("-v", "--check-links")
     result.assert_outcomes(passed=8, failed=4)
 
+@skip_pywin32
 def test_rst(testdir):
     testdir.copy_example('rst.rst')
     result = testdir.runpytest("-v", "--check-links")
@@ -20,5 +21,5 @@ def test_link_ext(testdir):
     testdir.copy_example('linkcheck.ipynb')
     testdir.copy_example('rst.rst')
     testdir.copy_example('markdown.md')
-    result = testdir.runpytest("-v", "--check-links", "--links-ext=.md,rst")
-    result.assert_outcomes(passed=15, failed=6)
+    result = testdir.runpytest("-v", "--check-links", "--links-ext=md,ipynb")
+    result.assert_outcomes(passed=11, failed=7)
diff --git a/test/test_cli.py b/test/test_cli.py
new file mode 100644
index 0000000..099c6e6
--- /dev/null
+++ b/test/test_cli.py
@@ -0,0 +1,24 @@
+import subprocess
+import pytest
+
+
+def test_cli_meta():
+    assert subprocess.call(["pytest-check-links", "--version"]) == 0
+    assert subprocess.call(["pytest-check-links", "--help"]) == 0
+
+
+@pytest.mark.parametrize("example,rc,expected,unexpected", [
+    ["httpbin.md", 0, [" 6 passed"], [" failed", " warning"]],
+    ["rst.rst", 1, [" 2 failed", " 7 passed"], [" warning"]]
+])
+def test_cli_pass(testdir, example, rc, expected, unexpected):
+    testdir.copy_example(example)
+    testdir.copy_example("setup.cfg")
+    proc = subprocess.Popen(["pytest-check-links"], stdout=subprocess.PIPE)
+    stdout, stderr = proc.communicate()
+    summary = stdout.decode('utf-8').strip().splitlines()[-1]
+    assert rc == proc.returncode
+    for ex in expected:
+        assert ex in summary, stdout.decode('utf-8')
+    for unex in unexpected:
+        assert unex not in summary, stdout.decode('utf-8')