Repository: pandas-dev/pandas
Pull request: Use tempfile for creating a file in home directory for a IO test
- [X] closes #40091
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [X] whatsnew entry


Problem statement:
BUG: IO unit test creates file at home directory
- [X] I have checked that this issue has not already been reported.
- [X] I have confirmed this bug exists on the latest version of pandas.
- [X] (optional) I have confirmed this bug exists on the master branch of pandas.
---

The unit test at `pandas/tests/io/test_common.py` produces a file using a hardcoded name as shown below:
```python
filename = path_type("~/sometest")
with icom.get_handle(filename, "w") as handles:
  assert os.path.isabs(handles.handle.name)
  assert os.path.expanduser(filename) == handles.handle.name
```

#### Problem description
A developer may encounter an empty file "sometest" appearing in their home directory (after running `pytest`). If the developer is unaware of that, then it may raise some wrong security concerns on their side. Also, in the extreme scenario that the developer has a file at the exact same path, then its content may be replaced.

#### Expected Output
I would expect the file produced by the unit test to be temporary and get deleted afterwards. Also, that the filename is generated rather than hardcoded.

Hints from issue discussion:
@chrispe Thanks for the bug report. I can confirm this on master that a sometest file is created in the home directory(I'm testing this on MacOS). I think we should at least check for the existence of a file before overwriting and for sure clean up the file after we're done. cc @twoertwein 
That test should use:

```py
import pandas._testing as tm

with tm.ensure_clean() as path:
    ....
```
In this test case we want to ensure that the path is able to expand correctly (i.e. `~/` is converted to an absolute path to home directory) right? However, the `tm.ensure_clean` does not allow creating a temp file in a custom directory. Should we then try to expand its functionality or directly use the module of `tempfile`?
you are right! If that is the only place where a file in "~"needs to be cleaned up, I wouldn't extend `tm.ensure_clean`.
By a quick look at the rest of the tests, it seems like this is the only hardcoded case. If you agree, I can continue with a PR to use the `tempfile` module for the file creation.
> By a quick look at the rest of the tests, it seems like this is the only hardcoded case. If you agree, I can continue with a PR to use the `tempfile` module for the file creation.

Yes, please go ahead :)

Commit message: Use 'with' for tmp and use read mode for handle

Patch to review:
diff --git a/pandas/tests/io/test_common.py b/pandas/tests/io/test_common.py
index db742fb69dd10..ad14c9b5020fb 100644
--- a/pandas/tests/io/test_common.py
+++ b/pandas/tests/io/test_common.py
@@ -7,8 +7,10 @@
     StringIO,
 )
 import mmap
+import ntpath
 import os
 from pathlib import Path
+import tempfile
 
 import pytest
 
@@ -119,10 +121,11 @@ def test_infer_compression_from_path(self, extension, expected, path_type):
     @pytest.mark.parametrize("path_type", [str, CustomFSPath, Path])
     def test_get_handle_with_path(self, path_type):
         # ignore LocalPath: it creates strange paths: /absolute/~/sometest
-        filename = path_type("~/sometest")
-        with icom.get_handle(filename, "w") as handles:
-            assert os.path.isabs(handles.handle.name)
-            assert os.path.expanduser(filename) == handles.handle.name
+        with tempfile.NamedTemporaryFile(dir=os.path.expanduser("~/")) as tmp:
+            filename = "~/" + ntpath.basename(tmp.name)
+            with icom.get_handle(filename, "r") as handles:
+                assert os.path.isabs(handles.handle.name)
+                assert os.path.expanduser(filename) == handles.handle.name
 
     def test_get_handle_with_buffer(self):
         input_buffer = StringIO()


Code review comments:
diff_hunk: @@ -119,10 +121,11 @@ def test_infer_compression_from_path(self, extension, expected, path_type):
     @pytest.mark.parametrize("path_type", [str, CustomFSPath, Path])
     def test_get_handle_with_path(self, path_type):
         # ignore LocalPath: it creates strange paths: /absolute/~/sometest
-        filename = path_type("~/sometest")
-        with icom.get_handle(filename, "w") as handles:
-            assert os.path.isabs(handles.handle.name)
-            assert os.path.expanduser(filename) == handles.handle.name
+        with tempfile.NamedTemporaryFile(dir=os.path.expanduser("~/")) as tmp:
path: pandas/tests/io/test_common.py
text: @user1:
use tm.ensure_clean

@author:
I agree, but current implementation of `tm.ensure_clean` doesn't support custom directories (which is needed in this test case). Should we then first extend that function?

@user1:
ok i guess this is fine. can you use pathlib.Path instead