Repository: jazzband/pip-tools

Issue:
pip-tools --upgrade --dry-run --output-file requirements.pc should not erase requirements.pc when --dry-run is specified
Describe the issue briefly here.

##### Environment Versions

1. OS Type

$ uname -v
Darwin Kernel Version 16.7.0: Thu Dec 20 21:53:35 PST 2018; root:xnu-3789.73.31~1/RELEASE_X86_64

2. Python version: `$ python -V`

$ python -V
Python 3.7.3

3. pip version:  `$ pip --version`

$ pip --version
pip 19.0.3[snip]

4. pip-tools version: `$ pip-compile --version`

$ pip-compile --version
pip-compile, version 3.8.0

##### Steps to replicate

1. Run pip-compile --output-file requirements.pc requirements.in
2. Run pip-compile --upgrade --dry-run --output-file requirements.pc
3. requirements.pc is recreated as an empty file rather than no effects taking place via --dry-run

##### Expected result

requirements.pc should be left alone if --upgrade is used with --dry-run

##### Actual result

requirements.pc is replaced with an empty file

##### Commentary

I'd first like to say thank you to everyone for this wonderful tool. I appreciate the sheer amount of work and attention to detail. Apologies if this is a repeat issue; I did look to see if this was already reported.

I suspect this behavior is due to click's file handling behavior being left turned on even for --dry-run?

As for why I'm using this mode at all and why other people may not have seen this...

I'm trying to run the command to let our developers know what *would* be upgraded vs. the existing requirements.pc and my expectation is that I'd be able to use the --output-file - stdout function and pipe to diff vs. the existing requirements.pc file with no damage caused. BUT, because when run with --dry-run no output is created so stdout is empty. When attempting to use the existing file and perhaps run git diff to highlight differences, the file disappears (bad). Another option, and one I'll take for the time being, is to run the upgrade command without --run-run into a dummy file and diff that vs. the real one. Unless someone else has a better idea?

Thanks, again, and warm regards.


Discussion:
One thing to add is that we run pip-tools with --quiet to avoid the terminal noise. I should have put --quiet in the examples. I find automatically spewing to stdout kind of ill advised when --output-file is specified.
Hello @shipmints,

Thank you for the very detailed report and good catch! I guess that's because the lazy output file has been accidentally opened for some reason. Investigating.
Found it. If you add `--no-header` flag it won't erase output file:

```bash
$ pip-compile --upgrade --dry-run --output-file requirements.pc --no-header
```

Lazy output file opens when `pip-compile` tries to get output's file name on rendering this header:

```
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file=requirements.pc requirements.in
#
```

Here is the line causes the bug: https://github.com/jazzband/pip-tools/blob/de874f4eb159d5a64ac05e2d0e3bbf2f7e1f8d4d/piptools/utils.py#L341-L347

And here is the output file opens on [click's side](https://github.com/pallets/click/blob/4da5e93cede17262424671208799bc6921dcfa36/click/utils.py#L98
):
```python
70   class LazyFile(object):
         ...
97       def __getattr__(self, name):
98           return getattr(self.open(), name)
```
Unfortunately, there is no a proper test for `dry-run`, that could have helped to catch that bug in advance :(

Resolving patch:
diff --git a/piptools/utils.py b/piptools/utils.py
--- a/piptools/utils.py
+++ b/piptools/utils.py
@@ -6,6 +6,7 @@
 from itertools import chain, groupby
 
 import six
+from click.utils import LazyFile
 from six.moves import shlex_quote
 
 from ._compat import install_req_from_line
@@ -339,11 +340,7 @@ def get_compile_command(click_ctx):
             continue
 
         # Use a file name for file-like objects
-        if (
-            hasattr(value, "write")
-            and hasattr(value, "read")
-            and hasattr(value, "name")
-        ):
+        if isinstance(value, LazyFile):
             value = value.name
 
         # Convert value to the list


Test patch:
diff --git a/tests/test_cli_compile.py b/tests/test_cli_compile.py
--- a/tests/test_cli_compile.py
+++ b/tests/test_cli_compile.py
@@ -722,3 +722,74 @@ def test_pre_option(runner, cli_option, infile_option, expected_package):
 
     assert out.exit_code == 0, out.output
     assert expected_package in out.output.splitlines(), out.output
+
+
+@pytest.mark.parametrize(
+    "add_options",
+    [
+        [],
+        ["--output-file", "requirements.txt"],
+        ["--upgrade"],
+        ["--upgrade", "--output-file", "requirements.txt"],
+        ["--upgrade-package", "small-fake-a"],
+        ["--upgrade-package", "small-fake-a", "--output-file", "requirements.txt"],
+    ],
+)
+def test_dry_run_option(runner, add_options):
+    """
+    Tests pip-compile doesn't create requirements.txt file on dry-run.
+    """
+    with open("requirements.in", "w") as req_in:
+        req_in.write("small-fake-a\n")
+
+    out = runner.invoke(
+        cli, ["--dry-run", "--find-links", MINIMAL_WHEELS_PATH] + add_options
+    )
+
+    assert out.exit_code == 0, out.output
+    assert "small-fake-a==0.2" in out.output.splitlines()
+    assert not os.path.exists("requirements.txt")
+
+
+@pytest.mark.parametrize(
+    "add_options, expected_cli_output_package",
+    [
+        ([], "small-fake-a==0.1"),
+        (["--output-file", "requirements.txt"], "small-fake-a==0.1"),
+        (["--upgrade"], "small-fake-a==0.2"),
+        (["--upgrade", "--output-file", "requirements.txt"], "small-fake-a==0.2"),
+        (["--upgrade-package", "small-fake-a"], "small-fake-a==0.2"),
+        (
+            ["--upgrade-package", "small-fake-a", "--output-file", "requirements.txt"],
+            "small-fake-a==0.2",
+        ),
+    ],
+)
+def test_dry_run_doesnt_touch_output_file(
+    runner, add_options, expected_cli_output_package
+):
+    """
+    Tests pip-compile doesn't touch requirements.txt file on dry-run.
+    """
+    with open("requirements.in", "w") as req_in:
+        req_in.write("small-fake-a\n")
+
+    with open("requirements.txt", "w") as req_txt:
+        req_txt.write("small-fake-a==0.1\n")
+
+    before_compile_mtime = os.stat("requirements.txt").st_mtime
+
+    out = runner.invoke(
+        cli, ["--dry-run", "--find-links", MINIMAL_WHEELS_PATH] + add_options
+    )
+
+    assert out.exit_code == 0, out.output
+    assert expected_cli_output_package in out.output.splitlines()
+
+    # The package version must NOT be updated in the output file
+    with open("requirements.txt", "r") as req_txt:
+        assert "small-fake-a==0.1" in req_txt.read().splitlines()
+
+    # The output file must not be touched
+    after_compile_mtime = os.stat("requirements.txt").st_mtime
+    assert after_compile_mtime == before_compile_mtime