Repository: pypa/setuptools_scm

Issue:
LookupError: setuptools-scm was unable to detect version (different case)
This is similar (but not the same as) https://github.com/pypa/setuptools_scm/issues/278

I have a private github repo hosting a number of packages, each of which is in an immediate subdirectory of the repo root.  Each project has a similar setup.py.  In some cases projects have dependencies on other projects from the same repo (but for the purposes of this problem I'm just considering one that does not).

The directory structure (from the repo root looks like this):

```
root
  c12e-data
    setup.py
    ...
```

The setup.py has content:

```
import os
from setuptools import setup

setup(
    name            = 'c12e_data',
    packages        = ['c12e.data'],
    use_scm_version = {
        'root'       : '..',
        'relative_to': os.path.dirname(__file__)
    },

    install_requires = [
        'typing_extensions'
    ],

    tests_require = [
        'hypothesis',
        'pytest >= 3.8.1, < 4.0',
        'c12e_testing'
    ],

    extras_require = {
        's3': ['boto3 >= 1.0.0, < 2.0.0']
    },

    setup_requires = [
        'setuptools_scm',
        'pytest-runner'
    ]
)
```

A tagged version of the repo has existed for several months and that tag has not changed.  The tag concerned is `v1.0.4`.  In 'real life' installation is via a conda `environment.yml` which pins `pip` and `setuptools`:

```
name: c12e-test
channels:
  - conda-forge
  - pytorch
  - defaults
dependencies:
  - pip=19.0.1
  - python=3.6.8
  - setuptools=40.7.3
  - toolz=0.9.0
  - scikit-learn
  - pip:
    - git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
```

However for the purposes of minimal reproduction of the issue I can reproduce with just a `pip` install in an environment already set up with those versions of `pip` and `setuptools` using:

```
pip install  git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
```

This fails with:

```
pip install  git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
Collecting git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
  Cloning ssh://git@github.com/CognitiveScale/c12e-python.git (to revision v1.0.4) to /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-zr1k52z7
Enter passphrase for key '/Users/sdraper/.ssh/github':
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-zr1k52z7/c12e_data/setup.py", line 28, in <module>
        'pytest-runner'
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/site-packages/setuptools/dist.py", line 447, in __init__
        k: v for k, v in attrs.items()
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/distutils/dist.py", line 281, in __init__
        self.finalize_options()
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/site-packages/setuptools/dist.py", line 739, in finalize_options
        ep.load()(self, ep.name, value)
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-zr1k52z7/c12e_data/.eggs/setuptools_scm-3.3.3-py3.6.egg/setuptools_scm/integration.py", line 17, in version_keyword
        dist.metadata.version = get_version(**value)
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-zr1k52z7/c12e_data/.eggs/setuptools_scm-3.3.3-py3.6.egg/setuptools_scm/__init__.py", line 150, in get_version
        parsed_version = _do_parse(config)
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-zr1k52z7/c12e_data/.eggs/setuptools_scm-3.3.3-py3.6.egg/setuptools_scm/__init__.py", line 113, in _do_parse
        "use git+https://github.com/user/proj.git#egg=proj" % config.absolute_root
    LookupError: setuptools-scm was unable to detect version for '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T'.

    Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

    For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
```

I have run this with verbose output from `pip` and debug output from `setuptools`, but it wasn't hugely revealing (to me anyway).  Specifically:

```
SETUPTOOLS_SCM_DEBUG=1 pip install -vvv git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
Created temporary directory: /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-ephem-wheel-cache-p27ef1wu
Created temporary directory: /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-tracker-d0aoaudl
Created requirements tracker '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-tracker-d0aoaudl'
Created temporary directory: /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-install-d3l9rxw6
Collecting git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
  Created temporary directory: /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma
  Cloning ssh://git@github.com/CognitiveScale/c12e-python.git (to revision v1.0.4) to /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma
  Running command git clone -q ssh://git@github.com/CognitiveScale/c12e-python.git /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma
Enter passphrase for key '/Users/sdraper/.ssh/github':
  Running command git show-ref v1.0.4
  86662033d860a213121fe4deee2d85f0d01a7f16 refs/tags/v1.0.4
  Running command git rev-parse HEAD
  b4f9b49838d97bc51ad86617f19ce53074d08907
  Running command git checkout -q 86662033d860a213121fe4deee2d85f0d01a7f16
  Added git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data to build tracker '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-tracker-d0aoaudl'
    Running setup.py (path:/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data/setup.py) egg_info for package from git+ssh://git@github.com/CognitiveScale/c12e-python.git@v1.0.4#subdirectory=c12e_data
    Running command python setup.py egg_info
    root '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data'
    root '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma'
    root '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T'
    looking for ep setuptools_scm.parse_scm /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T
    looking for ep setuptools_scm.parse_scm_fallback /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data
    found ep pip-egg-info = setuptools_scm.hacks:parse_pip_egg_info
    pip-egg-info /private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data/pip-egg-info []
    found ep setup.py = setuptools_scm.hacks:fallback_version
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data/setup.py", line 28, in <module>
        'pytest-runner'
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/site-packages/setuptools/dist.py", line 447, in __init__
        k: v for k, v in attrs.items()
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/distutils/dist.py", line 281, in __init__
        self.finalize_options()
      File "/Users/sdraper/miniconda3/envs/c12e-python-recommender-examples/lib/python3.6/site-packages/setuptools/dist.py", line 739, in finalize_options
        ep.load()(self, ep.name, value)
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data/.eggs/setuptools_scm-3.3.3-py3.6.egg/setuptools_scm/integration.py", line 17, in version_keyword
        dist.metadata.version = get_version(**value)
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data/.eggs/setuptools_scm-3.3.3-py3.6.egg/setuptools_scm/__init__.py", line 150, in get_version
        parsed_version = _do_parse(config)
      File "/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T/pip-req-build-iq2g3xma/c12e_data/.eggs/setuptools_scm-3.3.3-py3.6.egg/setuptools_scm/__init__.py", line 113, in _do_parse
        "use git+https://github.com/user/proj.git#egg=proj" % config.absolute_root
    LookupError: setuptools-scm was unable to detect version for '/private/var/folders/pd/0s5cjm_13ddd1h72c75dskrwsm34nk/T'.

    Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

    For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
Cleaning up...
```

I have also tried running the steps `pip` takes manually (create temp dir, clone, checkout tag, cd to subdirectory, then run `python setup.py egg_info` and this worked (and created the egg just fine)!

Also I am 99.99% sure that exactly the same `environment.yml` was working a few weeks ago (and the tag definitely has not changed).  Could there have been some transitive dependency change (in setuptools or pip) causing a regression or something?  I cannot think what else might have changed.

Discussion:

Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 12bf230..c022499 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -4,6 +4,8 @@ v5.0.0
 
 Breaking changes:
 * fix #339: strict errors on missing scms when  parsing a scm dir to avoid false version lookups
+* fix #337: if relative_to is a directory instead of a file,
+            consider it as direct target instead  of the containing folder and print a warning
 
 Bugfixes:
 
diff --git a/src/setuptools_scm/config.py b/src/setuptools_scm/config.py
index f0d9243..8b8fb8e 100644
--- a/src/setuptools_scm/config.py
+++ b/src/setuptools_scm/config.py
@@ -27,13 +27,24 @@ def _check_tag_regex(value):
 
 
 def _check_absolute_root(root, relative_to):
+    trace("l", repr(locals()))
     if relative_to:
         if os.path.isabs(root) and not root.startswith(relative_to):
             warnings.warn(
                 "absolute root path '%s' overrides relative_to '%s'"
                 % (root, relative_to)
             )
-        root = os.path.join(os.path.dirname(relative_to), root)
+        if os.path.isdir(relative_to):
+            warnings.warn(
+                "relative_to is expected to be a file,"
+                " its the directory %r\n"
+                "assuming the parent directory was passed" % (relative_to,)
+            )
+            trace("dir", relative_to)
+            root = os.path.join(relative_to, root)
+        else:
+            trace("file", relative_to)
+            root = os.path.join(os.path.dirname(relative_to), root)
     return os.path.abspath(root)
 
 
@@ -94,6 +105,7 @@ class Configuration(object):
         self._absolute_root = _check_absolute_root(self._root, value)
         self._relative_to = value
         trace("root", repr(self._absolute_root))
+        trace("relative_to", repr(value))
 
     @property
     def root(self):
@@ -104,6 +116,7 @@ class Configuration(object):
         self._absolute_root = _check_absolute_root(value, self._relative_to)
         self._root = value
         trace("root", repr(self._absolute_root))
+        trace("relative_to", repr(self._relative_to))
 
     @property
     def tag_regex(self):


Test patch:
diff --git a/testing/test_basic_api.py b/testing/test_basic_api.py
index 5111542..5e35a9e 100644
--- a/testing/test_basic_api.py
+++ b/testing/test_basic_api.py
@@ -90,10 +90,14 @@ def test_pretended(version, monkeypatch):
     assert setuptools_scm.get_version() == version
 
 
-def test_root_relative_to(monkeypatch, tmpdir):
-    assert_root(monkeypatch, tmpdir.join("alt").strpath)
-    __file__ = tmpdir.join("module/file.py").strpath
-    setuptools_scm.get_version(root="../alt", relative_to=__file__)
+def test_root_relative_to(monkeypatch, tmp_path):
+    assert_root(monkeypatch, str(tmp_path / "alt"))
+    module = tmp_path / "module/file.py"
+    module.parent.mkdir()
+    module.touch()
+    setuptools_scm.get_version(root="../alt", relative_to=str(module))
+    with pytest.warns(UserWarning, match="relative_to is expected to be a file.*"):
+        setuptools_scm.get_version(root="../alt", relative_to=str(module.parent))
 
 
 def test_dump_version(tmpdir):