Repository: pypa/pipenv

Issue:
--pre behaves weirdly in various commands
Be sure to check the existing issues (both open and closed!).

Describe the issue briefly here.

- `pipenv lock` should respect `allow_prereleases` in existing Pipfile
- `pipenv install packagename` fails to pass `--pre` either from command line or from Pipfile into `pipenv lock`, causes the command to fail during locking
  - Either we pass `--pre` or have` allow_prereleases` in the Pipfile, as long as there's package name in the cli arguments, it does not work, because of this code fails to passing `pre=pre`, to my guess: https://github.com/pypa/pipenv/blob/master/pipenv/core.py#L2005
- `pipenv update --pre --outdated` not respecting the `--pre` flag, removing `--outdated` works

------------

##### Expected result

None of the things above works when there's a prerelease dependency in the Pipfile, for example, `proto-google-datastore-v1beta3`

##### Actual result

They should all work (IMO, at least for the last two)

##### Steps to replicate

Pipfile is as follows:

```
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
"proto-google-datastore-v1beta3" = "*"

[dev-packages]
ipdb = "*"

[requires]
python_version = "3.6"

[pipenv]
allow_prereleases = true
```

Running any of the 3 commands above should fail with this Pipfile with the error

```
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches proto-google-datastore-v1beta3
Tried: 0.0.2rc0, 0.0.3rc0, 1.0.0b0, 1.0.0b1, 1.0.0b2
```


Discussion:
Minimal example:
```
[packages]
proto-google-datastore-v1beta3 = "*"

[pipenv]
allow_prereleases = true
```
Can't replicate
What is this package name ‘proto-google-datastore-v1beta3’? It does not follow Pep440 ?
@gsemet `proto-google-datastore-v1beta3` is a valid package name, and actually does exist. The `v1beta3` is part of the package name, not version.
Yes but does they use a prerelease version or normal version?
Ok i see they are in the logs. Pipenv update —pre should work.
I think @shivawu is talking specifically in the context of installing the package without the prerelease flag, then adding it to the pipfile, then attempting to update it by passing the package name specifically. Is that right? If not can you give us step by step instructions to reproduce?
No, sorry for the misunderstanding. I'm talking about 3 different scenarios, as there are 3 items in the original post.

The step to reproduce, take this exact Pipfile (without a Pipfile.lock).
```
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
"proto-google-datastore-v1beta3" = "*"

[dev-packages]
ipdb = "*"

[requires]
python_version = "3.6"

[pipenv]
allow_prereleases = true
```

Problem 1: Run `pipenv install ipython` (ipython is any package), it would successfully add the package into the Pipfile, but fails at the locking stage, because `pre` flag is not successfully propagated.

```
 !  ~/D/T/pipenv  pipenv install ipython                                46.7s  五  3/23 10:05:19 2018
Installing ipython…
Collecting ipython
  Using cached ipython-6.2.1-py3-none-any.whl
Collecting decorator (from ipython)
  Using cached decorator-4.2.1-py2.py3-none-any.whl
Collecting prompt-toolkit<2.0.0,>=1.0.4 (from ipython)
  Using cached prompt_toolkit-1.0.15-py3-none-any.whl
Collecting traitlets>=4.2 (from ipython)
  Using cached traitlets-4.3.2-py2.py3-none-any.whl
Collecting pygments (from ipython)
  Using cached Pygments-2.2.0-py2.py3-none-any.whl
Collecting jedi>=0.10 (from ipython)
  Using cached jedi-0.11.1-py2.py3-none-any.whl
[[source]]
Collecting simplegeneric>0.8 (from ipython)
Collecting pickleshare (from ipython)
  Using cached pickleshare-0.7.4-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=18.5 in /Users/wucy/.local/share/virtualenvs/pipenv-JzloF-4y/lib/python3.6/site-packages (from ipython)
Collecting appnope; sys_platform == "darwin" (from ipython)
  Using cached appnope-0.1.0-py2.py3-none-any.whl
Collecting pexpect; sys_platform != "win32" (from ipython)
  Using cached pexpect-4.4.0-py2.py3-none-any.whl
Collecting wcwidth (from prompt-toolkit<2.0.0,>=1.0.4->ipython)
  Using cached wcwidth-0.1.7-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9.0 in /Users/wucy/.local/share/virtualenvs/pipenv-JzloF-4y/lib/python3.6/site-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython)
Collecting ipython-genutils (from traitlets>=4.2->ipython)
  Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl
Collecting parso==0.1.1 (from jedi>=0.10->ipython)
  Using cached parso-0.1.1-py2.py3-none-any.whl
Collecting ptyprocess>=0.5 (from pexpect; sys_platform != "win32"->ipython)
  Using cached ptyprocess-0.5.2-py2.py3-none-any.whl
Installing collected packages: decorator, wcwidth, prompt-toolkit, ipython-genutils, traitlets, pygments, parso, jedi, simplegeneric, pickleshare, appnope, ptyprocess, pexpect, ipython
Successfully installed appnope-0.1.0 decorator-4.2.1 ipython-6.2.1 ipython-genutils-0.2.0 jedi-0.11.1 parso-0.1.1 pexpect-4.4.0 pickleshare-0.7.4 prompt-toolkit-1.0.15 ptyprocess-0.5.2 pygments-2.2.0 simplegeneric-0.8.1 traitlets-4.3.2 wcwidth-0.1.7

Adding ipython to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches proto-google-datastore-v1beta3
Tried: 0.0.2rc0, 0.0.3rc0, 1.0.0b0, 1.0.0b1, 1.0.0b2
```

2. Again with this Pipfile, no Pipfile.lock, running `pipenv lock`, I'm saying lock command should respect allow_prerelease flag

3. Agin with this Pipfile, no Pipfile.lock, running the following command fails as well
```
 ~/D/T/pipenv  pipenv update --pre --outdated                            882ms  五  3/23 10:07:53 2018

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches proto-google-datastore-v1beta3
Tried: 0.0.2rc0, 0.0.3rc0, 1.0.0b0, 1.0.0b1, 1.0.0b2
```
Ping, anything I can help with on the clarification or anything else?
@shivawu, all of the 3 issues have the same root cause: `lock` doesn't respect `allow_prereleases` setting.

I don't know if it is intended, but I can send a PR for this.
No definitely not intended. PR welcome!

Resolving patch:
diff --git a/pipenv/cli.py b/pipenv/cli.py
--- a/pipenv/cli.py
+++ b/pipenv/cli.py
@@ -503,9 +503,6 @@ def lock(
 
     # Ensure that virtualenv is available.
     core.ensure_project(three=three, python=python)
-    # Load the --pre settings from the Pipfile.
-    if not pre:
-        pre = core.project.settings.get('pre')
     if requirements:
         core.do_init(dev=dev, requirements=requirements)
     core.do_lock(
@@ -739,9 +736,6 @@ def update(
                 crayons.white('.', bold=True),
             )
         )
-        # Load the --pre settings from the Pipfile.
-        if not pre:
-            pre = core.project.settings.get('pre')
         core.do_lock(
             verbose=verbose, clear=clear, pre=pre, keep_outdated=keep_outdated
         )
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -1011,6 +1011,8 @@ def do_lock(
     from notpip._vendor.distlib.markers import Evaluator
     allowed_marker_keys = ['markers'] + [k for k in Evaluator.allowed_values.keys()]
     cached_lockfile = {}
+    if not pre:
+        pre = project.settings.get('allow_prereleases')
     if keep_outdated:
         if not project.lockfile_exists:
             click.echo(
@@ -1643,7 +1645,6 @@ def warn_in_virtualenv():
 
 def ensure_lockfile(keep_outdated=False):
     """Ensures that the lockfile is up–to–date."""
-    pre = project.settings.get('allow_prereleases')
     if not keep_outdated:
         keep_outdated = project.settings.get('keep_outdated')
     # Write out the lockfile if it doesn't exist, but not if the Pipfile is being ignored
@@ -1660,9 +1661,9 @@ def ensure_lockfile(keep_outdated=False):
                 ),
                 err=True,
             )
-            do_lock(pre=pre, keep_outdated=keep_outdated)
+            do_lock(keep_outdated=keep_outdated)
     else:
-        do_lock(pre=pre, keep_outdated=keep_outdated)
+        do_lock(keep_outdated=keep_outdated)
 
 
 def do_py(system=False):
@@ -2017,8 +2018,6 @@ def do_uninstall(
         system = True
     # Ensure that virtualenv is available.
     ensure_project(three=three, python=python)
-    # Load the --pre settings from the Pipfile.
-    pre = project.settings.get('allow_prereleases')
     package_names = (package_name,) + more_packages
     pipfile_remove = True
     # Un-install all dependencies, if --all was provided.
@@ -2085,7 +2084,7 @@ def do_uninstall(
             project.remove_package_from_pipfile(package_name, dev=True)
             project.remove_package_from_pipfile(package_name, dev=False)
     if lock:
-        do_lock(system=system, pre=pre, keep_outdated=keep_outdated)
+        do_lock(system=system, keep_outdated=keep_outdated)
 
 
 def do_shell(three=None, python=False, fancy=False, shell_args=None):
@@ -2183,7 +2182,7 @@ def inline_activate_virtualenv():
         activate_this = which('activate_this.py')
         with open(activate_this) as f:
             code = compile(f.read(), activate_this, 'exec')
-            exec (code, dict(__file__=activate_this))
+            exec(code, dict(__file__=activate_this))
     # Catch all errors, just in case.
     except Exception:
         click.echo(


Test patch:
diff --git a/tests/pypi/sqlalchemy/SQLAlchemy-1.2.0b3.tar.gz b/tests/pypi/sqlalchemy/SQLAlchemy-1.2.0b3.tar.gz
new file mode 100644
Binary files /dev/null and b/tests/pypi/sqlalchemy/SQLAlchemy-1.2.0b3.tar.gz differ
diff --git a/tests/test_pipenv.py b/tests/test_pipenv.py
--- a/tests/test_pipenv.py
+++ b/tests/test_pipenv.py
@@ -953,6 +953,25 @@ def test_complex_lock_with_vcs_deps(self, pip_src_dir, pypi):
             assert 'python_dateutil' not in lock['default']
             assert 'python_dateutil' not in lock['develop']
 
+    @pytest.mark.lock
+    @pytest.mark.requirements
+    def test_lock_with_prereleases(self, pypi):
+
+        with PipenvInstance(pypi=pypi) as p:
+            with open(p.pipfile_path, 'w') as f:
+                contents = """
+[packages]
+sqlalchemy = "==1.2.0b3"
+
+[pipenv]
+allow_prereleases = true
+                """.strip()
+                f.write(contents)
+
+            c = p.pipenv('lock')
+            assert c.return_code == 0
+            assert p.lockfile['default']['sqlalchemy']['version'] == '==1.2.0b3'
+
     @pytest.mark.lock
     @pytest.mark.requirements
     @pytest.mark.complex