Repository: gristlabs/asttokens (language: python)

Issue:
CI is absurdly slow
CI seems to take ~30 minutes to run `pytest`, which locally takes about 10 seconds. Do we have any idea what's causing this?


Discussion / hints:

Fix patch:
diff --git a/.coveragerc b/.coveragerc
new file mode 100644
index 0000000..f5a8aae
--- /dev/null
+++ b/.coveragerc
@@ -0,0 +1,3 @@
+[run]
+branch = True
+include = asttokens/*
diff --git a/README.rst b/README.rst
index edbf579..796f2b5 100644
--- a/README.rst
+++ b/README.rst
@@ -79,4 +79,5 @@ To contribute:
 
 3. Run tests in your current interpreter with the command ``pytest`` or ``python -m pytest``.
 4. Run tests across all supported interpreters with the ``tox`` command. You will need to have the interpreters installed separately. We recommend ``pyenv`` for that. Use ``tox -p auto`` to run the tests in parallel.
-5. By default certain tests which take a very long time to run are skipped, but they are run on travis CI. To run them locally, set the environment variable ``ASTTOKENS_SLOW_TESTS``. For example run ``ASTTOKENS_SLOW_TESTS=1 tox`` to run the full suite of tests.
+5. By default certain tests which take a very long time to run are skipped, but they are run in CI.
+   These are marked using the ``pytest`` marker ``slow`` and can be run on their own with ``pytest -m slow`` or as part of the full suite with ``pytest -m ''``.
diff --git a/setup.cfg b/setup.cfg
index 2419f6e..5a45658 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -47,6 +47,8 @@ test =
     astroid >=1, <2; python_version < "3"
     astroid >=2, <4; python_version >= "3"
     pytest
+    pytest-cov
+    pytest-xdist
 
 [options.package_data]
 asttokens = py.typed
@@ -59,4 +61,6 @@ universal=1
 
 
 [tool:pytest]
-addopts = --disable-warnings --ignore=tests/testdata
+addopts = --disable-warnings --ignore=tests/testdata --strict-markers -m 'not slow'
+markers =
+  slow: marks tests as slow (deselected by default)
diff --git a/tox.ini b/tox.ini
index 79078e3..c215be9 100644
--- a/tox.ini
+++ b/tox.ini
@@ -10,5 +10,3 @@ envlist = py{27,35,36,37,38,39,310,311,py,py3}
 commands = pytest {posargs}
 deps =
     .[test]
-passenv =
-    ASTTOKENS_SLOW_TESTS


Test patch:
diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml
index 3dfc306..41caa80 100644
--- a/.github/workflows/build-and-test.yml
+++ b/.github/workflows/build-and-test.yml
@@ -43,7 +43,6 @@ jobs:
             astroid-version: '<3'
 
     env:
-      ASTTOKENS_SLOW_TESTS: 1
       COVERALLS_PARALLEL: true
 
     steps:
@@ -81,11 +80,17 @@ jobs:
         # pypy < 3.8 very doesn't work
         # 2.7 is tested separately in mypy-py2, as we need to run mypy under Python 3.x
 
-      - name: Test with pytest
+      - name: Fast tests with coverage
         run: |
-          coverage run --branch --include='asttokens/*' -m pytest --junitxml=./rspec.xml
+          pytest --cov -n auto -m "not slow"
           coverage report -m
 
+      - name: Slow tests without coverage
+        # Python 2.7 doesn't have any slow tests
+        if: ${{ !contains(fromJson('["2.7", "pypy-2.7"]'), matrix.python-version) }}
+        run: |
+          pytest -n auto -m slow
+
       - name: Collect coverage results
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/tests/test_mark_tokens.py b/tests/test_mark_tokens.py
index 1ba01f2..e6a7768 100644
--- a/tests/test_mark_tokens.py
+++ b/tests/test_mark_tokens.py
@@ -5,6 +5,7 @@
 import inspect
 import io
 import os
+import pytest
 import re
 import sys
 import textwrap
@@ -630,18 +631,18 @@ def test_complex_slice_and_parens(self):
     self.create_mark_checker(source)
 
   if six.PY3:
+    @pytest.mark.slow
     def test_sys_modules(self):
       """
       Verify all nodes on source files obtained from sys.modules.
+
       This can take a long time as there are many modules,
-      so it only tests all modules if the environment variable
-      ASTTOKENS_SLOW_TESTS has been set.
+      so is marked as a slow test and must be explicitly selected
+      for running.
       """
       from .test_astroid import AstroidTreeException
 
       modules = list(sys.modules.values())
-      if not os.environ.get('ASTTOKENS_SLOW_TESTS'):
-        modules = modules[:20]
 
       start = time()
       for module in modules: