Repository: falconry/falcon

Issue:
3.1.0: examples directory not contained in pypi sdist tarball
Hi! I'm packaging falcon for Arch Linux. When trying to update to 3.1.0 I attempted to run the included tests.
This fails unfortunately, because pytest can not find required files contained in the examples directory:

```
============================= test session starts ==============================
platform linux -- Python 3.10.4, pytest-7.1.1, pluggy-1.0.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /build/python-falcon/src/falcon-3.1.0, configfile: setup.cfg
plugins: asyncio-0.17.1, anyio-3.5.0
asyncio: mode=strict
collecting ... collected 3384 items / 1 error

==================================== ERRORS ====================================
________________ ERROR collecting tests/test_things_example.py _________________
ImportError while importing test module '/build/python-falcon/src/falcon-3.1.0/tests/test_things_example.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_things_example.py:1: in <module>
    from examples.things import app
E   ModuleNotFoundError: No module named 'examples'
=============================== warnings summary ===============================
../../../../usr/lib/python3.10/site-packages/testtools/distutilscmd.py:7
  /usr/lib/python3.10/site-packages/testtools/distutilscmd.py:7: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    from distutils.core import Command

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR tests/test_things_example.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 4.65s ==========================
```


Discussion:
Hi @dvzrv!
I see what you mean. Unfortunately, it seems to be an unintended regression from https://github.com/falconry/falcon/pull/1917 where the `examples` module was prevented from being installed as such (a bug independently discovered by IIRC Gentoo and Debian maintainers), which effectively removed it from the `sdist` too.

How hard is it to override this on your side? Or are you dependent on us cutting `3.1.1` ASAP to solve this?

I know that, for instance, Gentoo grabs the tarball from GitHub instead along the lines of https://github.com/falconry/falcon/archive/3.1.0.tar.gz, maybe this approach could work for you as well for the time being?
(And IIRC others like Fedora and Debian maintain a Git repo with own patches which they sync to the upstream (us) when the time comes.)

Alternatively, maybe it would be possible to simply exclude `tests/test_things_example.py` on your side as a temporary workaround?
Whoever takes this: note that while it is trivial to add this to `MANIFEST.in`, it looks like we also need a CI gate where Falcon tests are run from the `sdist` tarball as opposed to a local checkout managed by _Actions_.
@vytas7 thank you for the fast reply! I can revert to using the github tarball for now.

Resolving patch:
diff --git a/tools/tox_sdist.py b/tools/tox_sdist.py
new file mode 100755
--- /dev/null
+++ b/tools/tox_sdist.py
@@ -0,0 +1,72 @@
+#!/usr/bin/env python
+
+import argparse
+import atexit
+import logging
+import pathlib
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+
+logging.basicConfig(
+    format='[tox_sdist.py] %(asctime)s [%(levelname)s] %(message)s', level=logging.INFO
+)
+HERE = pathlib.Path(__file__).resolve().parent
+DIST = HERE.parent / 'dist'
+
+
+def build_sdist():
+    subprocess.check_call((sys.executable, 'setup.py', 'sdist'))
+    archives = sorted(DIST.glob('*.tar.gz'), key=lambda p: p.stat().st_mtime)
+    assert archives, 'no sdist archives found, something is wrong!'
+    archive = archives[-1]
+    logging.info(f'Built sdist: {archive}.')
+    return archive
+
+
+def extract_sdist(archive, target):
+    with tarfile.open(archive) as targz:
+        targz.extractall(target)
+
+    content = target / archive.with_suffix('').stem
+    assert content.is_dir, 'no extracted directory found, something is wrong!'
+    logging.info(f'Extracted {archive} to {content}.')
+    return content
+
+
+def exec_tox(tox_ini_dir):
+    logging.info(f'Running tox in {tox_ini_dir}...')
+    subprocess.check_call(('tox', '--version'))
+    subprocess.check_call(('tox',), cwd=tox_ini_dir)
+
+
+def _cleanup(target):
+    shutil.rmtree(target)
+    logging.info(f'Removed temporary directory {target}.')
+
+
+def main():
+    description = 'Build, extract sdist to a temp path, and run tox within.'
+
+    parser = argparse.ArgumentParser(description=description)
+    parser.add_argument(
+        '-p', '--path', help='path to extract sdist to (default: create temp)'
+    )
+    args = parser.parse_args()
+
+    target = args.path
+    if not target:
+        target = tempfile.mkdtemp(prefix='falcon-')
+        atexit.register(_cleanup, target)
+    target = pathlib.Path(target)
+    logging.info(f'Using target path: {target}.')
+
+    archive = build_sdist()
+    extracted = extract_sdist(archive, target)
+    exec_tox(extracted)
+
+
+if __name__ == '__main__':
+    main()


Test patch: