Repository: conda/conda-build

Issue:
Build with bdist_conda leaves *.conda_trash files in the package
### Actual Behavior
I have a python package that I am packaging with `python setup.py bdist_conda`, which has worked flawlessly in the past. I have just noticed that now, on building the package, the finished package ends up with lots and lots of files that do not belong there, and that have a `conda_trash` prefix, **sometimes multiple times**. 
For example, I have a file called `python.exe.conda_trash.conda_trash` in the root of my .tar file. 
Compared to an earlier version, the yellow marked files/directory are new/should not be there:
![image](https://user-images.githubusercontent.com/327442/54831199-364aa380-4cba-11e9-8ac5-f0f08316a50b.png)

Is this some build clean-up phase failing?

During build, I also notice lots of
```
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\pyexpat.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\select.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\unicodedata.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_asyncio.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_bz2.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_decimal.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_hashlib.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_lzma.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_multiprocessing.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_overlapped.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_socket.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_sqlite3.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\DLLs\_ssl.pyd.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\Library\bin\libssl-1_1-x64.dll.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\Library\bin\sqlite3.dll.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\Library\bin\zlib.dll.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\python.exe.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\python3.dll.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\python36.dll.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\HOST_P~1.1-P\vcruntime140.dll.conda_trash.conda_trash.conda_trash.conda_trash - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\pyexpat.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\select.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\unicodedata.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_asyncio.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_bz2.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_decimal.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_hashlib.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_lzma.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_multiprocessing.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_overlapped.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_socket.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_sqlite3.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\DLLs\_ssl.pyd - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\Library\bin\libssl-1_1-x64.dll - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\Library\bin\sqlite3.dll - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\Library\bin\zlib.dll - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\python.exe - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\python3.dll - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\python36.dll - Access is denied.
C:\Continuum\miniconda3\conda-bld\bdist_conda_1553265400948\_TEST_~1\vcruntime140.dll - Access is denied.
```
but as far as I know these have always been there, and not made a problem.
I also notice multiple occurrences of
```
WARNING:conda_build.build:failed to open python.exe.conda_trash.conda_trash for detecting prefix.  Skipping it.
```
(with different files).

### Expected Behavior
The package should just have the `info` and `Lib` folder with all the related files therein, and no `conda_trash` files.

### Steps to Reproduce
The package is not public, I have not yet tried to reproduce independently with some minimal python package.

##### `Output of conda info`
<!-- Between the ticks below, paste the output of 'conda info' -->
```
     active environment : base
    active env location : C:\Continuum\miniconda3
            shell level : 1
       user config file : C:\Users\buchner\.condarc
 populated config files : C:\Users\buchner\.condarc
          conda version : 4.6.8
    conda-build version : 3.17.8
         python version : 3.6.8.final.0
       base environment : C:\Continuum\miniconda3  (writable)
           channel URLs : <a local channel>/win-64
                           <a local channel>/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/win-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
                          https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : C:\Continuum\miniconda3\pkgs
                          C:\Users\buchner\.conda\pkgs
                          C:\Users\buchner\AppData\Local\conda\conda\pkgs
       envs directories : C:\Continuum\miniconda3\envs
                          C:\Users\buchner\.conda\envs
                          C:\Users\buchner\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.6.8 requests/2.21.0 CPython/3.6.8 Windows/10 Windows/10.0.17763
          administrator : False
             netrc file : None
           offline mode : False
```
This is on Windows with miniconda, python 3.6, conda 4.6.8, conda-build 3.17.8, conda-env 2.6.0, conda-verify 3.1.1.



Discussion:
I only noticed this when I suddenly had trouble installing packages because of errors like
```
CondaVerificationError: The package for <snip> located at C:\Continuum\anaconda3\pkgs\<snip>
appears to be corrupted. The path 'DLLs/_asyncio.pyd.conda_trash.conda_trash'
specified in the package manifest cannot be found.
```
BTW, which manifest is this talking about? I can't find a file or content with "manifest" in it in the package.
@msarahan you seem to be involved with the (presumably) related code places, do you know what could be going wrong here?
.conda_trash files are created when conda tries to remove folders, but the files within are still being used by something in Windows.  I don't know why this has broken.  I also don't have time to investigate further right now.  If you come up with a fix, a PR would be most welcome.
I see. I mean, it would be fine if these files get left behind during a build, but I am assuming they should be filtered out when creating the package archive, right?
Yes, they should.  See the filter_files function in conda_build/utils.py.  Adding trash files there should be easy.  conda_trash files are pretty new, and conda-build just doesn't know about them yet.

Yes, it's a bug.  Yes, I'd like to fix it.  No, we can't dedicate resources right now, but we'll get to it when we can.

Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -1365,7 +1365,8 @@ def filter_files(files_list, prefix, filter_patterns=('(.*[\\\\/])?\.git[\\\\/].
                                                       '(.*[\\\\/])?\.git$',
                                                       '(.*)?\.DS_Store.*',
                                                       '.*\.la$',
-                                                      'conda-meta.*')):
+                                                      'conda-meta.*',
+                                                      '.*\.conda_trash(?:_\d+)*$')):
     """Remove things like the .git directory from the list of files to be copied"""
     for pattern in filter_patterns:
         r = re.compile(pattern)


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -197,7 +197,8 @@ def test_expand_globs(testing_workdir):
 def test_filter_files():
     # Files that should be filtered out.
     files_list = ['.git/a', 'something/.git/a', '.git\\a', 'something\\.git\\a',
-                 'file.la', 'something/file.la']
+                  'file.la', 'something/file.la', 'python.exe.conda_trash', 
+                  'bla.dll.conda_trash_1', 'bla.dll.conda_trash.conda_trash']
     assert not utils.filter_files(files_list, '')
 
     # Files that should *not* be filtered out.
@@ -205,7 +206,8 @@ def test_filter_files():
     #    lib/python3.4/site-packages/craftr/stl/craftr.utils.git/Craftrfile
     files_list = ['a', 'x.git/a', 'something/x.git/a',
                   'x.git\\a', 'something\\x.git\\a', 'something/.gitmodules',
-                  'some/template/directory/.gitignore', 'another.lab']
+                  'some/template/directory/.gitignore', 'another.lab',
+                  'miniconda_trashcan.py', 'conda_trash_avoider.py']
     assert len(utils.filter_files(files_list, '')) == len(files_list)