Repository: conda/conda

Issue:
IsADirectoryError
**I'm submitting a...**
  - [X] bug report

## Current Behavior
When installing certain packages it will break with IsADirectoryError
```
<<HTTPS 200 OK
< Accept-Ranges: bytes
< Content-Disposition: attachment; filename="six-1.11.0-py35h423b573_1.tar.bz2"; filename*=UTF-8''six-1.11.0-py35h423b573_1.tar.bz2
< Content-Type: application/x-tar
< Date: Mon, 29 Jan 2018 11:43:42 GMT
< ETag: "576d38abfcb7ceb3b323b802b1fec665"
< Last-Modified: Sat, 21 Oct 2017 18:55:29 GMT
< Server: AmazonS3
< x-amz-id-2: IGiKj9lk7QKmYRYyZCVaUEM4Ax19T3slC8e1+ZVXTR3gT5Hg9UNd/8CxoS9yDWZt5PdfORpg9mo=
< x-amz-request-id: 8A668C11FD79A607
< Content-Length: 21870
< Elapsed: 00:00.106578

TRACE conda.core.path_actions:execute(1238): extracting /opt/conda/pkgs/six-1.11.0-py35h423b573_1.tar.bz2 => /opt/conda/pkgs/six-1.11.0-py35h423b573_1
TRACE conda.gateways.disk.update:rename(60): renaming /opt/conda/pkgs/six-1.11.0-py35h423b573_1 => /opt/conda/pkgs/six-1.11.0-py35h423b573_1.c~
DEBUG conda.gateways.disk.create:extract_tarball(132): extracting /opt/conda/pkgs/six-1.11.0-py35h423b573_1.tar.bz2
  to /opt/conda/pkgs/six-1.11.0-py35h423b573_1
TRACE conda.gateways.disk.create:write_as_json_to_file(61): writing json to file /opt/conda/pkgs/six-1.11.0-py35h423b573_1/info/repodata_record.json
TRACE conda.gateways.disk.create:write_as_json_to_file(61): writing json to file /opt/conda/pkgs/six-1.11.0-py35h423b573_1/info/repodata_record.json
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/six-1.11.0-py35h423b573_1.tar.bz2.c~
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/six-1.11.0-py35h423b573_1.c~
DEBUG conda.common.signals:signal_handler(56): de-registering handler for Signals.SIGIOT
DEBUG conda.common.signals:signal_handler(56): de-registering handler for Signals.SIGINT
DEBUG conda.common.signals:signal_handler(56): de-registering handler for Signals.SIGTERM
DEBUG conda.common.signals:signal_handler(56): de-registering handler for Signals.SIGQUIT
Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/site-packages/conda/exceptions.py", line 772, in __call__
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.5/site-packages/conda_env/cli/main.py", line 74, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "/opt/conda/lib/python3.5/site-packages/conda_env/cli/main_update.py", line 107, in execute
    installer.install(prefix, specs, args, env)
  File "/opt/conda/lib/python3.5/site-packages/conda_env/installers/conda.py", line 38, in install
    pfe.execute()
  File "/opt/conda/lib/python3.5/site-packages/conda/core/package_cache.py", line 590, in execute
    raise CondaMultiError(exceptions)
conda.CondaMultiError: [Errno 21] Is a directory: '/opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0'
[Errno 21] Is a directory: '/opt/conda/pkgs/libedit-3.1-heed3624_0'
```

### Steps to Reproduce
- run the continuumio/miniconda3 docker image
- run `conda update conda`
- create a file test.yaml with the following contents:
```
name: test
dependencies:
- lxml==3.7.2
- anaconda::redis-py==2.10.5
- beautifulsoup4==4.5.3
- feedparser==5.2.1
- html5lib==0.999
- w3lib==1.16.0
- clinicalgraphics::libgcrypt11=1
- python=3.5
```
- run `conda env update -f test.yaml`


## Expected Behavior
It should install without problems


## Environment Information
```
     active environment : None
       user config file : /root/.condarc
 populated config files : 
          conda version : 4.4.8
    conda-build version : not installed
         python version : 3.6.2.final.0
       base environment : /opt/conda  (writable)
           channel URLs : https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.4.8 requests/2.18.4 CPython/3.6.2 Linux/4.4.0-112-generic debian/8 glibc/2.19
                UID:GID : 0:0
             netrc file : None
           offline mode : False

```
</p></details>


<details open><summary><code>`conda config --show-sources`</code></summary><p>

```

```
</p></details>


<details><summary><code>`conda list --show-channel-urls`</code></summary><p>
<!-- between the ticks below, paste the output of 'conda list --show-channel-urls' -->

```
# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
asn1crypto                0.22.0           py36h265ca7c_1    defaults
ca-certificates           2017.08.26           h1d4fec5_0    defaults
certifi                   2017.7.27.1      py36h8b7b77e_0    defaults
cffi                      1.10.0           py36had8d393_1    defaults
chardet                   3.0.4            py36h0f667ec_1    defaults
conda                     4.4.8                    py36_0    defaults
conda-env                 2.6.0                h36134e3_1    defaults
cryptography              2.0.3            py36ha225213_1    defaults
idna                      2.6              py36h82fb2a8_1    defaults
libedit                   3.1                  heed3624_0    defaults
libffi                    3.2.1                h4deb6c0_3    defaults
libgcc-ng                 7.2.0                hcbc56d2_1    defaults
libstdcxx-ng              7.2.0                h24385c6_1    defaults
ncurses                   6.0                  h06874d7_1    defaults
openssl                   1.0.2l               h9d1a558_3    defaults
pip                       9.0.1            py36h30f8307_2    defaults
pycosat                   0.6.3            py36h0a5515d_0    defaults
pycparser                 2.18             py36hf9f622e_1    defaults
pyopenssl                 17.2.0           py36h5cc804b_0    defaults
pysocks                   1.6.7            py36hd97a5b1_1    defaults
python                    3.6.2               h02fb82a_12    defaults
readline                  7.0                  hac23ff0_3    defaults
requests                  2.18.4           py36he2e5f8d_1    defaults
ruamel_yaml               0.11.14          py36ha2fb22d_2    defaults
setuptools                36.5.0           py36he42e2e1_0    defaults
six                       1.10.0           py36hcac75e4_1    defaults
sqlite                    3.20.1               h6d8b0f3_1    defaults
tk                        8.6.7                h5979e9b_1    defaults
urllib3                   1.22             py36hbe7ace6_0    defaults
wheel                     0.29.0           py36he7f4e38_1    defaults
xz                        5.2.3                h2bcbf08_1    defaults
yaml                      0.1.7                h96e3832_1    defaults
zlib                      1.2.11               hfbfcf68_1    defaults

```
</p></details>


Discussion:
I am also seeing this with conda-build.
For context, with conda-build, it happens very frequently with the cross-platform build.  So, on our linux-64 docker image, when we run with CONDA_SUBDIR=linux-32 and ``linux32 conda-build``, we can get this error right away.  I'm less sure about how to reproduce with the OP's report.  I know that @nehaljwani has tried, and it's tough to get to happen reliably.  I think that's because it depends on the state of the package cache.  You must have the correct initial state (something already existing there) to get this problem to happen.
This seems to be the same problem in my case, if run the command again, it seems to work. We have frozen conda to an old version as this was crashing our CI pipelines.
Do you think this is related to https://github.com/conda/conda/issues/6631?  That issue isn't new; it was present in conda 4.3, and probably 4.2.  But something in conda 4.4 could be helping to exacerbate it.
It seems to be unrelated, also this bug doesn't exist in 4.4.7. 
To reproduce this, exact steps:
```
# Spawn the container
sudo docker run --rm -it continuumio/miniconda3:4.3.27 /bin/bash

# Inside the container
cat <<EOF > ~/test.yaml 
name: test
dependencies:
- anaconda::ca-certificates==2017.08.26
EOF

# Update conda to 4.4.9
conda install -y conda=4.4.9

# Trigger the problem
conda env update -f ~/test.yaml 
Solving environment: done

Downloading and Extracting Packages
ca-certificates 2017.08.26: ############################5                   |  75% 

IsADirectoryError(21, 'Is a directory')
```
However, if I switch my docker daemon configuration from (overlay2/aufs):
```
{
  "storage-driver": "overlay2"
}
```
To:
```
{
  "storage-driver": "devicemapper"
}
```

Then the problem goes away. 
Logs:
[log_overlay2.txt](https://github.com/conda/conda/files/1716159/log_overlay2.txt)
[log_devicemapper.txt](https://github.com/conda/conda/files/1716160/log_devicemapper.txt)

But, that doesn't mean that this is not a bug in Conda. Because, if I switch the update from conda 4.4.9 to 4.4.7 (as @pvanderlinden has pointed out), the problem doesn't occur, even in the case of overlay2.

The difference in verbose logs is:
4.4.7:
```
TRACE conda.core.path_actions:execute(1238): extracting /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0
TRACE conda.gateways.disk.update:rename(50): renaming /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
DEBUG conda.core.path_actions:execute(1251): Invalid cross-device link on rename /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0
DEBUG conda.gateways.disk.create:extract_tarball(131): extracting /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2
  to /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0
TRACE conda.gateways.disk.create:write_as_json_to_file(60): writing json to file /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0/info/repodata_record.json
TRACE conda.gateways.disk.create:write_as_json_to_file(60): writing json to file /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0/info/repodata_record.json
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2.c~
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
TRACE conda.gateways.disk.delete:rm_rf(49): rm_rf failed. Not a link, file, or directory: /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
```
For conda > 4.4.7:
```
TRACE conda.core.path_actions:execute(1238): extracting /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0
TRACE conda.gateways.disk.update:rename(60): renaming /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
TRACE conda.gateways.disk.update:rename(67): Could not rename do to errno [18]. Falling back to copy/remove.
TRACE conda.gateways.disk.create:create_hard_link_or_copy(201): creating hard link /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
INFO conda.gateways.disk.create:create_hard_link_or_copy(204): hard link failed, so copying /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
TRACE conda.gateways.disk.create:_do_copy(251): copying /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0 => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.c~
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0
TRACE conda.core.path_actions:reverse(1206): moving /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2.c~ => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2
TRACE conda.gateways.disk.delete:rm_rf(29): rm_rf /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2
TRACE conda.gateways.disk.update:rename(60): renaming /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2.c~ => /opt/conda/pkgs/ca-certificates-2017.08.26-h1d4fec5_0.tar.bz2
DEBUG conda.core.package_cache:execute(586): IsADirectoryError(21, 'Is a directory')
NoneType: None
```
If I revert this commit: 0b57e1ea4ff49b82df3bcd02408368cf182a3aba , then the problem mentioned in this issue goes away, but of course, that's not the right fix.
Okay, seems like the [function in that commit ](https://github.com/conda/conda/blob/4.4.9/conda/gateways/disk/update.py#L47-L53)was written keeping files in mind, not directories!
So, the IsADirectoryError exception is coming from https://github.com/conda/conda/blob/4.4.9/conda/gateways/disk/create.py#L259

Resolving patch:
diff --git a/conda/gateways/disk/update.py b/conda/gateways/disk/update.py
--- a/conda/gateways/disk/update.py
+++ b/conda/gateways/disk/update.py
@@ -7,10 +7,11 @@
 from os import rename as os_rename, utime
 from os.path import dirname, isdir
 import re
+from shutil import move
 
 from . import exp_backoff_fn, mkdir_p, mkdir_p_sudo_safe
 from .delete import rm_rf
-from .link import islink, lexists
+from .link import lexists
 from ...common.compat import on_win
 from ...common.path import expand
 from ...exceptions import NotWritableError
@@ -44,15 +45,6 @@ def update_file_in_place_as_binary(file_full_path, callback):
             fh.close()
 
 
-def _copy_then_remove(source_path, destination_path):
-    from .create import copy, create_hard_link_or_copy
-    if islink(source_path):
-        copy(source_path, destination_path)
-    else:
-        create_hard_link_or_copy(source_path, destination_path)
-    rm_rf(source_path)
-
-
 def rename(source_path, destination_path, force=False):
     if lexists(destination_path) and force:
         rm_rf(destination_path)
@@ -62,10 +54,13 @@ def rename(source_path, destination_path, force=False):
             os_rename(source_path, destination_path)
         except EnvironmentError as e:
             if e.errno in (EINVAL, EXDEV):
-                # see https://github.com/conda/conda/issues/6711
-                log.trace("Could not rename do to errno [%s]. Falling back to copy/remove.",
-                          e.errno)
-                _copy_then_remove(source_path, destination_path)
+                # https://github.com/conda/conda/issues/6811
+                # https://github.com/conda/conda/issues/6711
+                log.trace("Could not rename %s => %s due to errno [%s]. Falling back"
+                          " to copy/unlink", source_path, destination_path, e.errno)
+                # https://github.com/moby/moby/issues/25409#issuecomment-238537855
+                # shutil.move() falls back to copy+unlink
+                move(source_path, destination_path)
             else:
                 raise
     else:


Test patch: