Repository: spyder-ide/spyder

Issue:
Bootstrap incorrectly detects pylsp's install status
## Description

### What steps will reproduce the problem?

<!--- You can use Markdown here --->

Starting Spyder from bootstrap
```
$ python bootstrap.py
```
results in 
```
2022-10-05 13:43:39,493 [INFO] [InstallDevRepos] -> Installing 'python-lsp-server' from source in editable mode.
```
even when `python-lsp-server` is already installed in editable mode.

It seems that `p == dist or p in dist.parents` is not sufficient to properly detect whether `python-lsp-server` is installed in editable mode.

## Versions

* Spyder version: 5.4.0.dev0 ab2322dc7 (conda)
* Python version: 3.10.6 64-bit
* Qt version: 5.15.4
* PyQt5 version: 5.15.7
* Operating System: Darwin 21.6.0

### Dependencies

```
# Mandatory:
applaunchservices >=0.3.0     :  0.3.0 (OK)
atomicwrites >=1.2.0          :  1.4.1 (OK)
chardet >=2.0.0               :  5.0.0 (OK)
cloudpickle >=0.5.0           :  2.2.0 (OK)
cookiecutter >=1.6.0          :  2.1.1 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.0.2 (OK)
IPython >=7.31.1;<8.0.0       :  7.33.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.16.0 (OK)
keyring >=17.0.0              :  23.9.3 (OK)
nbconvert >=4.0               :  7.1.0 (OK)
numpydoc >=0.6.0              :  1.4.0 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.2 (OK)
pygments >=2.0                :  2.13.0 (OK)
pylint >=2.5.0;<3.0           :  2.15.3 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.5.0;<1.6.0          :  1.5.0 (OK)
pylsp_black >=1.2.0           :  1.2.1 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.3 (OK)
qstylizer >=0.1.10            :  0.2.2 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.3.2;<5.4.0      :  5.4.0.dev0 (OK)
qtpy >=2.1.0                  :  2.2.1 (OK)
rtree >=0.9.7                 :  1.0.0 (OK)
setuptools >=49.6.0           :  65.4.1 (OK)
sphinx >=0.6.6                :  5.2.3 (OK)
spyder_kernels >=2.3.3;<2.4.0 :  2.4.0.dev0 (OK)
textdistance >=4.2.0          :  4.5.0 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.9 (OK)
zmq >=22.1.0                  :  24.0.1 (OK)

# Optional:
cython >=0.21                 :  0.29.32 (OK)
matplotlib >=3.0.0            :  3.6.0 (OK)
numpy >=1.7                   :  1.23.3 (OK)
pandas >=1.1.1                :  1.5.0 (OK)
scipy >=0.17.0                :  1.9.1 (OK)
sympy >=0.7.3                 :  1.11.1 (OK)
```



Discussion:
Pinging @CAM-Gerlach
@CAM-Gerlach, I don't know if this is related, but when Spyder and external-deps are installed in editable mode, `spyder` won't start _without_ bootstrap.
```
$ spyder
Traceback (most recent call last):
  File "/Users/rclary/opt/mambaforge/envs/spy-test/lib/python3.10/site-packages/pkg_resources/__init__.py", line 581, in _build_master
    ws.require(__requires__)
  File "/Users/rclary/opt/mambaforge/envs/spy-test/lib/python3.10/site-packages/pkg_resources/__init__.py", line 909, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/Users/rclary/opt/mambaforge/envs/spy-test/lib/python3.10/site-packages/pkg_resources/__init__.py", line 800, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (pyflakes 2.4.0 (/Users/rclary/opt/mambaforge/envs/spy-test/lib/python3.10/site-packages), Requirement.parse('pyflakes<2.6.0,>=2.5.0; extra == "all"'), {'python-lsp-server'})
```
This is only an issue if `python-lsp-server` is installed in editable mode. Note that `<env>/bin/spyder` uses entry-points to launch spyder, while bootstrap calls `spyder.app.start.main` directly.
> It seems that p == dist or p in dist.parents is not sufficient to properly detect whether python-lsp-server is installed in editable mode.

For some reason, `distribution("python-lsp-server").__path` is still in `site-packages` rather than the normal editable path, which is correctly returned by `pip list`. There are two possibilities I can immediately think of—either this has to do with it being installed as a PEP 660 editable wheel with support recently added by Setuptools, or it has something to do with `setuptools_scm` with lsp uses (or perhaps more likely, the build time dependency versions this forces, again including new versions of Setuptools with PEP 660 support). I will need to dig into this more, but I'm just too tired tonight (er, this morning) at the moment...

>  I don't know if this is related, but when Spyder and external-deps are installed in editable mode, spyder won't start without bootstrap.

That's the same basic issue as we saw several times before: Spyder's specified pinned deps have a version conflict in dev mode with that specified by pylsp, and due to the old-style Setuptools entry point script wrappers in legacy pre-PEP 660 editable mode (which calling `spyder.app.start.main` directly bypasses) and possibly an interaction with the `pkg_resources` use within Spyder itself (at least, I believe we saw that before) an error is raised.

Loosening the `pyflakes` req in Spyder's `setup.py`, at least for dev builds, should fix it (as we are doing with others already). If not, I can dig into it more.
> Loosening the pyflakes req in Spyder's setup.py, at least for dev builds, should fix it (as we are doing with others already). If not, I can dig into it more.

We don't depend on Pyflakes directly, so I don't understand what you mean by this.
> We don't depend on Pyflakes directly, so I don't understand what you mean by this.

Sorry, it was 4:30 in the morning and I was falling asleep at my desk—yeah, that requirement is in `python-lsp-server`, not Spyder itself; that is evidently required by `python-lsp-server` but doesn't get updated (even though it seems it does reinstall). I'll test this further and see what I can figure out.
This is driving me crazy, so I'm going to fix it.

Resolving patch:
diff --git a/install_dev_repos.py b/install_dev_repos.py
--- a/install_dev_repos.py
+++ b/install_dev_repos.py
@@ -38,12 +38,21 @@
         continue
 
     try:
-        dist = distribution(p.name)._path
+        dist = distribution(p.name)
     except PackageNotFoundError:
         dist = None
         editable = None
     else:
-        editable = (p == dist or p in dist.parents)
+        editable = (p == dist._path or p in dist._path.parents)
+
+        # This fixes detecting that PyLSP was installed in editable mode under
+        # some scenarios.
+        # Fixes spyder-ide/spyder#19712
+        if p.name == 'python-lsp-server':
+            for f in dist.files:
+                if 'editable' in f.name:
+                    editable = True
+                    break
 
     REPOS[p.name] = {'repo': p, 'dist': dist, 'editable': editable}
 


Test patch: