Repository: pypa/pipenv

Issue:
Unable to install package from private repository
I am attempting to install from a private repository. The output indicates the package and dependencies were installed but when pipenv is "Locking [packages] dependencies…"

I receive an error indicating: pip.exceptions.InstallationError: Could not detect requirement name, please specify one with #egg=

##### Describe your environment

1. macOS 10.13.2
1. Python version: Python 3.6.3
1. Pipenv version: pipenv, version 9.0.3

##### Expected result

I am expecting for the package to install and the lockfile to be written.

##### Actual result
```
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Traceback (most recent call last):
  File "/Users/clayk/Library/Python/3.6/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/cli.py", line 1934, in install
    do_lock(system=system, pre=pre)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/cli.py", line 1102, in do_lock
    pre=pre
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/utils.py", line 545, in resolve_deps
    resolved_tree = actually_resolve_reps(deps, index_lookup, markers_lookup, project, sources, verbose, clear, pre)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/utils.py", line 470, in actually_resolve_reps
    constraint = pip.req.InstallRequirement.from_editable(dep[len('-e '):])
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/patched/pip/req/req_install.py", line 143, in from_editable
    editable_req, default_vcs)
  File "/Users/clayk/Library/Python/3.6/lib/python/site-packages/pipenv/patched/pip/req/req_install.py", line 1196, in parse_editable
    "Could not detect requirement name, please specify one with #egg="
pip.exceptions.InstallationError: Could not detect requirement name, please specify one with #egg=
```
##### Steps to replicate

execute pipenv install -e git+ssh://path/to/private/Redacted.git@v1.1.8#egg=privatepackagename


Discussion:
I just started seeing this problem as well, but when I update the package rather than when I'm installing it. 

In the `Pipfile` I have an entry like:
```
mypkg = {git = "file:///some/path/to/working/dir", editable = true, subdirectory = "inside/workdir"}
```
when I run `pipenv update mypkg` the package gets uninstalled and reinstalled correctly, but at the end `pipenv` tries to add a new entry using just part of the original one, and in the `Pipfile` I get:
```
mypkg = {git = "file:///some/path/to/working/dir", editable = true, subdirectory = "inside/workdir"}
"a65c66e" = {path = "git+file:///some/path/to/working/dir", editable = true}
```
As you can see there's no `#egg` because the `mypkg` name is lost, and no `subdirectory`.

I'm sure it used to work.

Now I have to manually remove that additional line to keep my `Pipfile` clean, fortunately the update works so I can keep on until this is fixed

I am also seeing similar problems.

When installing without the `-e` flag the package is installed, but if I `pipenv install` afterwards I get:

```
Installing dependencies from Pipfile.lock (5fd147)…
An error occurred while installing git+git@github.com/my/package/! Will try again.
Installing initially–failed dependencies…
Invalid requirement: 'git+git@github.com/my/package/'
It looks like a path. Does it exist ?
```

If I install the same package with the `-e` flag I get:

```
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
fatal: repository 'git@github.com/my/package.git' does not exist
Traceback (most recent call last):
  File "/home/henrique/miniconda3/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/cli.py", line 213, in install
    deploy=deploy
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/core.py", line 1795, in do_install
    do_lock(system=system, pre=pre)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/core.py", line 1113, in do_lock
    allow_global=system
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/utils.py", line 340, in resolve_deps
    resolved_tree, resolver = actually_resolve_reps(deps, index_lookup, markers_lookup, project, sources, verbose, clear, pre)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/utils.py", line 301, in actually_resolve_reps
    resolved_tree.update(resolver.resolve(max_rounds=PIPENV_MAX_ROUNDS))
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 102, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 200, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 274, in _iter_dependencies
    for dependency in self.repository.get_dependencies(ireq):
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 153, in get_dependencies
    result = reqset._prepare_file(self.finder, ireq)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/req/req_set.py", line 521, in _prepare_file
    req_to_install.update_editable(not self.is_download)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/req/req_install.py", line 587, in update_editable
    vcs_backend.export(self.source_dir)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/vcs/git.py", line 69, in export
    self.unpack(temp_dir)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/vcs/__init__.py", line 286, in unpack
    self.obtain(location)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/vcs/git.py", line 142, in obtain
    self.run_command(['clone', '-q', url, dest])
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/vcs/__init__.py", line 325, in run_command
    spinner)
  File "/home/henrique/miniconda3/lib/python3.6/site-packages/pipenv/patched/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "git clone -q git@github.com/my/package.git /tmp/pip-53zad75t-export" failed with error code 128 in None
```
@techalchemy 
To install a private github repository from pip the syntax is
`pip install git+ssh://git@github.com/username/repo.git@branch#egg=blah`

This worked properly with pipenv up to version 9
`pipenv install git+ssh://git@github.com/username/repo.git@branch#egg=blah`

but with pipenv 11 
`pipenv install git+ssh://git@github.com/username/repo.git@branch#egg=blah`
returns with the an error for
`git clone -q git@github.com/username/repo.git /tmp/temp`
it's because the correct syntax should be `git clone -q git@github.com:username/repo.git /tmp/temp` with `:` and not `/`
@bokhi 
You're close to the actual issue, but your proposed fix is incorrect. The problem is not the slash, but the fact that the leading `ssh://` is omitted, causing pipenv to prompt for the ssh password for user git in my case.

I'm trying to install a package from a private Bitbucket repository from which you can only clone via SSH, not HTTPS. Pipenv tries to execute `git clone -q git@host:7999/project/repo.git` instead of `git clone -q ssh://git@host:7999/project/repo.git`.
For some reason, the stack trace is cut off. Here's what I get:

```
.py", line 162, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 199, in get_legacy_dependencies
    result = reqset._prepare_file(self.finder, ireq)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/req/req_set.py", line 521, in _prepare_file
    req_to_install.update_editable(not self.is_download)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/req/req_install.py", line 587, in update_editable
    vcs_backend.export(self.source_dir)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/vcs/git.py", line 69, in export
    self.unpack(temp_dir)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/vcs/__init__.py", line 286, in unpack
    self.obtain(location)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/vcs/git.py", line 142, in obtain
    self.run_command(['clone', '-q', url, dest])
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/vcs/__init__.py", line 325, in run_command
    spinner)
  File "/home/moritz/.local/lib/python2.7/site-packages/pipenv/patched/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
```
That’s not where the problem is but it is actually really helpful. If you use the `git+git@` syntax it should work until I get a fix in place 
@techalchemy How do you even specify the port with the syntax `git+git@host:project/repo.git`? I've tried a few variants and each of them results in a similar prompt for git's password (which should not happen). The only difference is that I don't get a stacktrace after entering something incorrect.
From https://pip.pypa.io/en/stable/reference/pip_install/#git

> Here are the supported forms:
> ...
> -e git+git@git.myproject.org:MyProject#egg=MyProject

Naturally if you use an ssh based approach you will need to have a password or a key pair for authentication. You can also use an http based approach which is not going to be impacted by this bug: `git+https://git.myhost.org/myrepo.git#egg=myrepo`

Please read the pip documentation if you want to know how to interact with vcs repositories. 
I read the pip documentation (the exact page you posted) and it doesn't cover how to specify the port when using this type of URI, which you've suggested as a workaround. That's why I asked.

As mentioned above, HTTP access is disabled in our Bitbucket instance. My public key is correctly registered with Bitbucket and normal interaction via git and installation of the package via pip itself work perfectly fine.

But no matter what I try, I cannot get pipenv to install the package. The fact that it's asking for a password for git makes me suspect that it's always connecting to the regular SSH server on port 22, not the Bitbucket SSH server on port 7999.

Sorry if what I wrote before wasn't clear enough.
Does simply adding the port like you would to a normal url not work?
I've tried the following variations:

 * `git+git@host:port/project/repo.git#egg=...`
 * `git+git@host:port:project/repo.git#egg=...`

Both don't work, since in this URL scheme, everything behind the first colon seems to be interpreted as the path.
A bit of additional info:

After the initial installation attempt (`pipenv install -e git+ssh://...`), the Pipfile contains the following line:
```
package = {git = "git@host:7999/project/repo.git", editable = true}
```

The step `Locking [packages] dependencies…` then immediately fails (i.e. asks for the password).

I tried to manually change the line to the following:
```
package = {git = "ssh://git@host:7999/project/repo.git", editable = true}
```

Running `pipenv install` after that change results in a partial success:
```
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (28b5f3)!
Installing dependencies from Pipfile.lock (28b5f3)…
```

This time, it at least installs all **dependencies** of the package correctly before failing to install the package itself (asking for the password). `pipenv run pip freeze` shows that the dependencies were indeed installed correctly.

Hope that helps track down the problem.

**Edit**: Manually adding "ssh://" in Pipfile.lock as well makes `pipenv install` succeed. I guess that's my workaround for now, then.
@Moritz90 thanks for the thorough debug, I'll get a fix together for this which will be much easier because of this info
@techalchemy will this fix also help with the original issue filed? It seems the discussion has shifted slightly regarding SSH specifics.
yes
To clarify for everyone since there is confusion apparently:

@Moritz90 @clayk - the `git+git@host` syntax _is the same thing_ as using `git+ssh://git@host` -- pip itself does a drop in of `ssh://` and we do this as well, this is the piece of code that currently is broken when you actually do use the more verbose format. 

@hcoura your issue is unrelated, you are not using the correct syntax.  You are using `git+git@github.com/my/package.git` and you need to be using a `:` instead of a `/`: `git+git@github.com:my/package.git`
@techalchemy I guess I wasn't explicit as I was using:

`pipenv install -e git+ssh://git@github.com/my/package`

Anyway, I did some more testing here(python 2.7.14 btw) and:

* `pipenv install -e git+ssh://git@github.com/my/package.git@v1.0.0#egg=my_package`

Install package but Lock fails, last message:
```
pip.exceptions.InstallationError: Command "git clone -q git@github.com/orchardmile/data-acquisition-core.git /tmp/pip-ri964xm6-export" failed with error code 128 in None
```
adds this line to Pipfile:
```
my_package = {git = "git@github.com/my/package.git.git", editable = true, ref = "v1.0.0"}
```

* `pipenv install -e git+ssh://git@github.com/my/package.git@v1.0.0#egg=my_package`

Install package but Lock fails, last message:
```
pip.exceptions.InstallationError: Command "git clone -q git@github.com/my/package.git /tmp/pip-ri964xm6-export" failed with error code 128 in None
```
adds this line to Pipfile:
```
my_package = {git = "git@github.com/my/package.git.git", editable = true, ref = "v1.0.0"}
```

* `git+git@github.com:my/package.git@v1.0.0#egg=my_package`

Installs fine and adds this line to Pipfile:
```
my_package = {git = "git@github.com:my/package.git.git", editable = true, ref = "v1.0.0"}
```

and everything works fine

Resolving patch:
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -1396,7 +1396,7 @@ def pip_install(
 
     # Install dependencies when a package is a VCS dependency.
     try:
-        req = get_requirement(package_name.split('--hash')[0].split('--trusted-host')[0]).vcs
+        req = get_requirement(package_name.split('--hash')[0].split('--trusted-host')[0], verbose=verbose).vcs
     except (pip._vendor.pyparsing.ParseException, ValueError) as e:
         click.echo('{0}: {1}'.format(crayons.red('WARNING'), e), err=True)
         click.echo(
@@ -2428,7 +2428,7 @@ def do_clean(
 
     installed_package_names = []
     for installed in installed_packages:
-        r = get_requirement(installed)
+        r = get_requirement(installed, verbose=verbose)
 
         # Ignore editable installations.
         if not r.editable:
diff --git a/pipenv/utils.py b/pipenv/utils.py
--- a/pipenv/utils.py
+++ b/pipenv/utils.py
@@ -45,7 +45,7 @@
 from piptools.resolver import Resolver
 from piptools.repositories.pypi import PyPIRepository
 from piptools.scripts.compile import get_pip_command
-from piptools import logging
+from piptools import logging as piptools_logging
 from piptools.exceptions import NoCandidateFound
 from pip.download import is_archive_file
 from pip.exceptions import DistributionNotFound
@@ -140,7 +140,7 @@ def get_requirement(dep):
     elif req.local_file and path and not req.vcs:
         req.path = path
         req.uri = None
-    elif req.vcs and req.uri and cleaned_uri and uri != req.uri:
+    elif req.vcs and req.uri and cleaned_uri and cleaned_uri != uri:
         req.uri = strip_ssh_from_git_uri(req.uri)
         req.line = strip_ssh_from_git_uri(req.line)
     req.editable = editable
@@ -314,6 +314,7 @@ class PipCommand(pip.basecommand.Command):
 
     if verbose:
         logging.log.verbose = True
+        piptools_logging.log.verbose = True
 
 
     resolved_tree = set()


Test patch: