Repository: pypa/pipenv

Issue:
2023.8.19 through to 2023.8.22 break editable package installation
### Issue description

Editable package installation appears to be broken from pipenv 2023.8.19 onwards.
For example, given a package in the directory `test-hello` under the pipenv root installing the package with `pipenv install -e ./test-hello` fails.

### Expected result

pipenv 2023.7.23 and prior versions would install editable packages in Pipfile and in the venv.

### Actual result

Here's a log of package installation across recent versions of pipenv.
[try_pipenv_versions.log](https://github.com/pypa/pipenv/files/12411124/try_pipenv_versions.log)

For example:
```
+ pipx install pipenv==2023.8.19
creating virtual environment...
installing pipenv from spec 'pipenv==2023.8.19'...
done! ✨ 🌟 ✨
  installed package pipenv 2023.8.19, installed using Python 3.8.10
  These apps are now globally available
    - pipenv
    - pipenv-resolver
+ git clean -dfx
Removing .venv/
Removing applications/test-hello/test_hello.egg-info/
Removing test-hello/test_hello.egg-info/
+ echo '=== Using pipenv version 2023.8.19 ==='
=== Using pipenv version 2023.8.19 ===
+ pipenv sync
Creating a virtualenv for this project...
Pipfile: /home/stewart/src/pipenv_editable_install_repro/Pipfile
Using /usr/bin/python3.8 (3.8.10) to create virtualenv...
created virtual environment CPython3.8.10.final.0-64 in 48ms
  creator CPython3Posix(dest=/home/stewart/src/pipenv_editable_install_repro/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/stewart/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.41.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/stewart/src/pipenv_editable_install_repro/.venv
Installing dependencies from Pipfile.lock (db4242)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!
+ install_test_and_uninstall_package ./test-hello
+ path=./test-hello
+ pipenv run pipenv install -e ./test-hello
Installing -e ./test-hello...
Resolving -e ./test-hello...
[31m[1mError:[0m Traceback (most recent call last):
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/routines/install.py", line 
279, in do_install
    added, cat, normalized_name = project.add_package_to_pipfile(
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/project.py", line 1159, in 
add_package_to_pipfile
    name, normalized_name, entry = self.generate_package_pipfile_entry(
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/project.py", line 1110, in 
generate_package_pipfile_entry
    req_name = determine_package_name(package)
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/utils/dependencies.py", line 
745, in determine_package_name
    raise ValueError(f"Could not determine package name from {package}")
ValueError: Could not determine package name from 
file:///file:///home/stewart/src/pipenv_editable_install_repro/test-hello

✘ Failed adding package to Pipfile
✔ Installation Succeeded
```

It's also interesting that installation fails, due to the mangled URI that starts with `file:///file:///`, but the final message is that installation succeeded.

### Steps to replicate

https://github.com/stewartmiles/pipenv_editable_install_repro contains a minimal test environment.
I generated the log file using:

```shell
git clone https://github.com/stewartmiles/pipenv_editable_install_repro
cd pipenv_editable_install_repro
./try_pipenv_versions.sh 2>&1 | tee ../try_pipenv_versions.log
```

-------------------------------------------------------------------------------

Please run `$ pipenv --support`, and paste the results here. Don't put backticks (`` ` ``) around it! The output already contains Markdown formatting.

<details><summary>$ pipenv --support</summary>

Pipenv version: `'2023.8.22'`

Pipenv location: `'/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv'`

Python location: `'/home/stewart/.local/pipx/venvs/pipenv/bin/python'`

OS Name: `'posix'`

User pip version: `'23.2.1'`

user Python installations found:


PEP 508 Information:

```
{'implementation_name': 'cpython',
 'implementation_version': '3.8.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.15.0-69-generic',
 'platform_system': 'Linux',
 'platform_version': '#76~20.04.1-Ubuntu SMP Mon Mar 20 15:54:19 UTC 2023',
 'python_full_version': '3.8.10',
 'python_version': '3.8',
 'sys_platform': 'linux'}
```

System environment variables:

  - `SHELL`
  - `PIPENV_VENV_IN_PROJECT`
  - `TERMCAP`
  - `SSH_AUTH_SOCK`
  - `WINDOW`
  - `SSH_AGENT_PID`
  - `PWD`
  - `LOGNAME`
  - `XDG_SESSION_TYPE`
  - `MOTD_SHOWN`
  - `HOME`
  - `LANG`
  - `LS_COLORS`
  - `SSH_CONNECTION`
  - `LESSCLOSE`
  - `XDG_SESSION_CLASS`
  - `TERM`
  - `LESSOPEN`
  - `USER`
  - `XLA_FLAGS`
  - `SHLVL`
  - `XDG_SESSION_ID`
  - `KUBECONFIG`
  - `LD_LIBRARY_PATH`
  - `XDG_RUNTIME_DIR`
  - `SSH_CLIENT`
  - `XDG_DATA_DIRS`
  - `PATH`
  - `STY`
  - `DBUS_SESSION_BUS_ADDRESS`
  - `SSH_TTY`
  - `USE_GKE_GCLOUD_AUTH_PLUGIN`
  - `OLDPWD`
  - `_`
  - `PIP_DISABLE_PIP_VERSION_CHECK`
  - `PYTHONDONTWRITEBYTECODE`
  - `PYTHONFINDER_IGNORE_UNSUPPORTED`

Pipenv–specific environment variables:

 - `PIPENV_VENV_IN_PROJECT`: `true`

Debug–specific environment variables:

  - `PATH`: `~/.local/bin:/home/stewart/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin`
  - `SHELL`: `/bin/bash`
  - `LANG`: `en_US.UTF-8`
  - `PWD`: `/home/stewart/src/pipenv_editable_install_repro`


---------------------------

Contents of `Pipfile` ('/home/stewart/src/pipenv_editable_install_repro/Pipfile'):

```toml
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.8"

```


Contents of `Pipfile.lock` ('/home/stewart/src/pipenv_editable_install_repro/Pipfile.lock'):

```json
{
    "_meta": {
        "hash": {
            "sha256": "7f7606f08e0544d8d012ef4d097dabdd6df6843a28793eb6551245d4b2db4242"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

```
</details>



Discussion:
Same here, I am working in Amplify and had to specify `pipenv==2023.7.23` in amplify.yml to make Amplify build work, otherwise (without specifying the version) it will raise 'Failed to install dependencies in ... Error: Command failed with exit code 1: pipenv install)'
Same issue here. Also breaking for `pipenv install "git+..."`.
@lewiswolf I believe that is a separate issue, there are a couple reports about the git+ vcs issue and an open PR that some have reported fix it. 

Regarding  this actual issue report, I think part of the problem (maybe not the whole problem) is the ast parser logic for the old setup.py file logic is not accounting for all cases.   I tried to correct for that some locally over lunch in the reproduction link (super helpful by the way) and was able to get it to install for the top level test-hello, but when I try the one in the applications sub-dir, I get a pathing issue.   

I'll be able to cut a release going  into late evening for the PR I have open, but I am not sure I have an immediate fix for what's going on with editable file installs in this issue report.
So I opened https://github.com/pypa/pipenv/pull/5861 which I did some testing on windows and linux and it appears to be past this issue, but what I noticed is on windows it will lock a relative path fine, but on linux in this branch its adding the full path to the Pipfile, and then locking that.  🤔 

`test-hello = {file = "file:/home/matteius/pipenv_editable_install_repro/applications/test-hello", editable=true}`

Resolving patch:
diff --git a/pipenv/project.py b/pipenv/project.py
--- a/pipenv/project.py
+++ b/pipenv/project.py
@@ -333,7 +333,8 @@ def get_hashes_from_remote_index_urls(self, ireq, source):
                 )
             return None
 
-    def get_file_hash(self, session, link):
+    @staticmethod
+    def get_file_hash(session, link):
         h = hashlib.new(FAVORITE_HASH)
         err.print(f"Downloading file {link.filename} to obtain hash...")
         with open_file(link.url, session) as fp:
@@ -1126,7 +1127,7 @@ def generate_package_pipfile_entry(self, package, pip_line, category=None):
         if extras:
             entry["extras"] = list(extras)
         if path_specifier:
-            entry["file"] = unquote(path_specifier)
+            entry["file"] = unquote(str(path_specifier))
         elif vcs_specifier:
             for vcs in VCS_LIST:
                 if vcs in package.link.scheme:
diff --git a/pipenv/utils/dependencies.py b/pipenv/utils/dependencies.py
--- a/pipenv/utils/dependencies.py
+++ b/pipenv/utils/dependencies.py
@@ -11,7 +11,7 @@
 from pathlib import Path
 from tempfile import NamedTemporaryFile, TemporaryDirectory
 from typing import Any, AnyStr, Dict, List, Mapping, Optional, Sequence, Union
-from urllib.parse import urljoin, urlparse, urlsplit, urlunsplit
+from urllib.parse import urlparse, urlsplit, urlunsplit
 
 from pipenv.patched.pip._internal.models.link import Link
 from pipenv.patched.pip._internal.network.download import Downloader
@@ -639,15 +639,12 @@ def find_package_name_from_directory(directory):
 def determine_path_specifier(package: InstallRequirement):
     if package.link:
         if package.link.scheme in ["http", "https"]:
-            path_specifier = package.link.url_without_fragment
-            return path_specifier
+            return package.link.url_without_fragment
         if package.link.scheme == "file":
             try:
-                path_specifier = os.path.relpath(package.link.file_path)
+                return Path(package.link.file_path).relative_to(Path.cwd()).as_posix()
             except ValueError:
-                # If os.path.relpath() fails, use the absolute path instead
-                path_specifier = os.path.abspath(package.link.file_path)
-            return path_specifier
+                return Path(package.link.file_path).as_posix()
 
 
 def determine_vcs_specifier(package: InstallRequirement):
@@ -900,13 +897,7 @@ def expansive_install_req_from_line(
     if expand_env:
         name = expand_env_variables(name)
 
-    if os.path.isfile(name) or os.path.isdir(name):
-        if not name.startswith("file:"):
-            # Make sure the path is absolute and properly formatted as a file: URL
-            absolute_path = os.path.abspath(name)
-            name = urljoin("file:", absolute_path)
-            name = "file:" + name
-
+    if editable or os.path.isdir(name):
         return install_req_from_editable(name, line_source)
 
     vcs_part = name
@@ -926,9 +917,7 @@ def expansive_install_req_from_line(
                 constraint=constraint,
                 user_supplied=user_supplied,
             )
-    if editable:
-        return install_req_from_editable(name, line_source)
-    if urlparse(name).scheme in ("http", "https", "file"):
+    if urlparse(name).scheme in ("http", "https", "file") or os.path.isfile(name):
         parts = parse_req_from_line(name, line_source)
     else:
         # It's a requirement
@@ -992,9 +981,9 @@ def install_req_from_pipfile(name, pipfile):
         else:
             req_str = f"{name}{extras_str}@ {req_str}"
     elif "path" in _pipfile:
-        req_str = f"{_pipfile['path']}{extras_str}"
+        req_str = str(Path(_pipfile["path"]).as_posix())
     elif "file" in _pipfile:
-        req_str = f"{_pipfile['file']}{extras_str}"
+        req_str = str(Path(_pipfile["file"]).as_posix())
     else:
         # We ensure version contains an operator. Default to equals (==)
         _pipfile["version"] = version = get_version(pipfile)
diff --git a/pipenv/utils/funktools.py b/pipenv/utils/funktools.py
--- a/pipenv/utils/funktools.py
+++ b/pipenv/utils/funktools.py
@@ -9,7 +9,7 @@
 import time
 import warnings
 from functools import partial
-from itertools import count, islice, tee
+from itertools import count, islice
 from typing import Any, Iterable
 
 DIRECTORY_CLEANUP_TIMEOUT = 1.0
@@ -62,18 +62,13 @@ def unnest(elem: Iterable) -> Any:
     """
 
     if isinstance(elem, Iterable) and not isinstance(elem, str):
-        elem, target = tee(elem, 2)
-    else:
-        target = elem
-    if not target or not _is_iterable(target):
-        yield target
-    else:
-        for el in target:
+        for el in elem:
             if isinstance(el, Iterable) and not isinstance(el, str):
-                el, el_copy = tee(el, 2)
-                yield from unnest(el_copy)
+                yield from unnest(el)
             else:
                 yield el
+    else:
+        yield elem
 
 
 def dedup(iterable: Iterable) -> Iterable:


Test patch:
diff --git a/tests/integration/test_install_uri.py b/tests/integration/test_install_uri.py
--- a/tests/integration/test_install_uri.py
+++ b/tests/integration/test_install_uri.py
@@ -187,7 +187,7 @@ def test_install_local_vcs_not_in_lockfile(pipenv_instance_pypi):
 def test_get_vcs_refs(pipenv_instance_private_pypi):
     with pipenv_instance_private_pypi() as p:
         c = p.pipenv(
-            "install -e git+https://github.com/benjaminp/six.git@1.9.0"
+            "install -e git+https://github.com/benjaminp/six.git@1.9.0#egg=six"
         )
         assert c.returncode == 0
         assert "six" in p.pipfile["packages"]