Repository: ibis-project/ibis

Issue:
Impala pandas_interop.py DataFrameWriter.write_csv() fails with PermissionError on Windows Platforms
pandas_interop.py DataFrameWriter.write_csv() fails with PermissionError on Windows Platforms due to how Pythons [tempfile.NamedTemporaryFile()](https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile) works. 

Specifically, if the OS is Windows and the NamedTemporaryFile delete parameter is True, the IO operation will fail, because Python will open the file and then Pandas will attempt to open the file again. See [this SO answer ](https://stackoverflow.com/a/23212515/3744998)for more information. 

I have replicated the issue manually and verified that setting the delete parameter to False resolves the issue. Perhaps we can add a platform check and explicitly set the delete parameter based on the platform? Then after we [exit the context statements scope](https://github.com/ibis-project/ibis/blob/master/ibis/impala/pandas_interop.py#L80), explicitly delete the temporary file. I imagine it could look something like the following.

``` python

    import os

    def write_csv(self, path):
        if os.name == 'nt':
            # Should probably think of a better name for this boolean
            delete = False
        else:
            delete = True

        with tempfile.NamedTemporaryFile(delete=delete) as f:
            tmp_name = f.name
            # Write the DataFrame to the temporary file path
            if options.verbose:
                util.log(
                    'Writing DataFrame to temporary file {}'.format(f.name)
                )

            self.df.to_csv(
                f.name,
                header=False,
                index=False,
                sep=',',
                quoting=csv.QUOTE_NONE,
                escapechar='\\',
                na_rep='#NULL',
            )
            f.seek(0)

            if options.verbose:
                util.log('Writing CSV to: {0}'.format(path))

            self.hdfs.put(path, f.name)

        if not delete:
            os.remove(tmp_name)

        return path
```


I don't know that much about the Ibis code base, so this is just a first pass. I hope it's helpful. 


Discussion:

Resolving patch:
diff --git a/ibis/impala/pandas_interop.py b/ibis/impala/pandas_interop.py
--- a/ibis/impala/pandas_interop.py
+++ b/ibis/impala/pandas_interop.py
@@ -13,8 +13,8 @@
 # limitations under the License.
 
 import csv
+import os
 import tempfile
-
 from posixpath import join as pjoin
 
 import ibis.common.exceptions as com
@@ -56,15 +56,21 @@ def write_temp_csv(self):
         return temp_hdfs_dir
 
     def write_csv(self, path):
-        with tempfile.NamedTemporaryFile() as f:
+        # Use a temporary dir instead of a temporary file
+        # to provide Windows support and avoid #2267
+        # https://github.com/ibis-project/ibis/issues/2267
+        with tempfile.TemporaryDirectory() as f:
             # Write the DataFrame to the temporary file path
+            tmp_file_path = os.path.join(f, 'impala_temp_file.csv')
             if options.verbose:
                 util.log(
-                    'Writing DataFrame to temporary file {}'.format(f.name)
+                    'Writing DataFrame to temporary directory {}'.format(
+                        tmp_file_path
+                    )
                 )
 
             self.df.to_csv(
-                f.name,
+                tmp_file_path,
                 header=False,
                 index=False,
                 sep=',',
@@ -72,12 +78,11 @@ def write_csv(self, path):
                 escapechar='\\',
                 na_rep='#NULL',
             )
-            f.seek(0)
 
             if options.verbose:
                 util.log('Writing CSV to: {0}'.format(path))
 
-            self.hdfs.put(path, f.name)
+            self.hdfs.put(path, tmp_file_path)
         return path
 
     def get_schema(self):


Test patch: