Repository: conda/conda-build

Issue:
CB deletes directory inside tarball with same name as package
Bob is back from vacation. As usual, Bob has been assigned the task to create a conda package for another team in his company. But being an unlucky fella, he stumbles upon yet another bug in Conda Build. His vacation mood vanishes in the first five minutes of work starting on a Wednesday.

The tarball of the other team has the following contents:
```
[bob@nutcasecompany testcase ]$ unzip -l mypkg.zip 
Archive:  mypkg.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2018-02-21 16:57   mypkg/
       74  2018-02-21 16:56   mypkg/libmypkg.c
        0  2018-02-21 16:53   mypkg/mypkg/
       22  2018-02-21 16:53   mypkg/mypkg/awesomeheader.h
---------                     -------
       96                     4 files
```

Bob has reduced the build script to the following example:
```
[bob@nutcasecompany testcase ]$ cat build.sh 
find $SRC_DIR -type f
if [[ ! -f "$SRC_DIR/mypkg/awesomeheader.h" ]]; then
    exit 1
fi
exit 0

[bob@nutcasecompany testcase ]$ cat meta.yaml 
package:
  name: mypkg
  version: 1.0.0

source:
  url: file:///tmp/testcase/mypkg.zip
```

Bob has attached the package [mypkg.zip](https://github.com/conda/conda-build/files/1743886/mypkg.zip) in the testcase.

Conda Build fails with:
```
[bob@nutcasecompany testcase ]$ conda build .
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for mypkg
INFO:conda_build.metadata:Attempting to finalize metadata for mypkg
BUILD START: [u'mypkg-1.0.0-0.tar.bz2']
Source cache directory is: /opt/conda/conda/root/conda-bld/src_cache
No hash (md5, sha1, sha256) provided.  Source download forced.  Add hash to recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided.  Source download forced.  Add hash to recipe to use source cache.
Downloading source to cache: mypkg.zip
Downloading file:///tmp/testcase/mypkg.zip
Success
Extracting download
source tree in: /opt/conda/conda/root/conda-bld/mypkg_1519214647277/work
/opt/conda/conda/root/conda-bld/mypkg_1519214647277/work/libmypkg.c
/opt/conda/conda/root/conda-bld/mypkg_1519214647277/work/conda_build.sh
Traceback (most recent call last):
  File "/opt/conda/conda/root/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 413, in main
    execute(sys.argv[1:])
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 404, in execute
    verify=args.verify)
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/api.py", line 193, in build
    need_source_download=need_source_download, config=config, variants=variants)
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/build.py", line 1944, in build_tree
    notest=notest,
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/build.py", line 1240, in build
    utils.check_call_env(cmd, env=env, cwd=src_dir)
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/utils.py", line 678, in check_call_env
    return _func_defaulting_env_to_os_environ(subprocess.check_call, *popenargs, **kwargs)
  File "/opt/conda/conda/root/lib/python2.7/site-packages/conda_build/utils.py", line 674, in _func_defaulting_env_to_os_environ
    return func(_args, **kwargs)
  File "/opt/conda/conda/root/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/opt/conda/conda/root/conda-bld/mypkg_1519214647277/work/conda_build.sh']' returned non-zero exit status 1
```

Somehow, CB eats up the file `awesomeheader.h`. Bob blames it on the function:
 [hoist_single_extracted_folder](https://github.com/conda/conda-build/blob/3.5.0/conda_build/source.py#L115)


Discussion:
Thanks for the test, @nehaljwani.  I'll incorporate this into the test suite.  This function has been a problem before - sorry we didn't have a better test already.
Previous test was at https://github.com/conda/conda-build/blob/master/tests/test_source.py#L106

It's not covering the situation where there's other files at the top level of the folder.  I'll add that to the test.
```
def test_hoist_same_name_with_toplevel_files(testing_workdir):
    testdir = os.path.join(testing_workdir, 'test', 'test')
    outer_dir = os.path.join(testing_workdir, 'test')
    os.makedirs(testdir)
    with open(os.path.join(testdir, 'somefile'), 'w') as f:
        f.write('weeeee')
    with open(os.path.join(outer_dir, 'some_other_file'), 'w') as f:
        f.write('weeeee')
    source.hoist_single_extracted_folder(testdir)
    assert os.path.isfile(os.path.join(testdir, 'somefile'))
    assert os.path.isfile(os.path.join(outer_dir, 'some_other_file'))
    assert os.path.isdir(testdir)
```

Resolving patch:
diff --git a/conda_build/source.py b/conda_build/source.py
--- a/conda_build/source.py
+++ b/conda_build/source.py
@@ -119,6 +119,7 @@ def hoist_single_extracted_folder(nested_folder):
     know exactly what that folder is called."""
     flist = os.listdir(nested_folder)
     parent = os.path.dirname(nested_folder)
+    # only hoist if the parent folder contains ONLY our nested folder
     nested_folders_to_remove = [nested_folder]
     for thing in flist:
         if not os.path.isdir(os.path.join(parent, thing)):
@@ -126,6 +127,11 @@ def hoist_single_extracted_folder(nested_folder):
         else:
             copy_into(os.path.join(nested_folder, thing), os.path.join(parent, thing))
             nested_folders_to_remove.append(os.path.join(nested_folder, thing))
+    # handle nested similar folder names
+    fn = os.path.basename(nested_folder)
+    if (os.path.join(nested_folder, fn) in nested_folders_to_remove and
+            nested_folder in nested_folders_to_remove):
+        nested_folders_to_remove.remove(nested_folder)
     for folder in nested_folders_to_remove:
         rm_rf(folder)
 


Test patch:
diff --git a/tests/test-recipes/metadata/hoist_source_multilevel/bld.bat b/tests/test-recipes/metadata/hoist_source_multilevel/bld.bat
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/hoist_source_multilevel/bld.bat
@@ -0,0 +1,3 @@
+if not exist "%SRC_DIR%\mypkg\awesomeheader.h" exit 1
+echo "found source files OK"
+exit 0
diff --git a/tests/test-recipes/metadata/hoist_source_multilevel/build.sh b/tests/test-recipes/metadata/hoist_source_multilevel/build.sh
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/hoist_source_multilevel/build.sh
@@ -0,0 +1,6 @@
+find $SRC_DIR -type f
+if [[ ! -f "$SRC_DIR/mypkg/awesomeheader.h" ]]; then
+    exit 1
+fi
+echo "found source files OK"
+exit 0
diff --git a/tests/test-recipes/metadata/hoist_source_multilevel/meta.yaml b/tests/test-recipes/metadata/hoist_source_multilevel/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/metadata/hoist_source_multilevel/meta.yaml
@@ -0,0 +1,6 @@
+package:
+  name: mypkg
+  version: 1.0.0
+
+source:
+  url: mypkg.zip
diff --git a/tests/test-recipes/metadata/hoist_source_multilevel/mypkg.zip b/tests/test-recipes/metadata/hoist_source_multilevel/mypkg.zip
new file mode 100644
Binary files /dev/null and b/tests/test-recipes/metadata/hoist_source_multilevel/mypkg.zip differ
diff --git a/tests/test_source.py b/tests/test_source.py
--- a/tests/test_source.py
+++ b/tests/test_source.py
@@ -116,12 +116,13 @@ def test_hoist_same_name(testing_workdir):
 
 def test_hoist_different_name(testing_workdir):
     testdir = os.path.join(testing_workdir, 'test')
-    os.makedirs(testdir)
-    with open(os.path.join(testdir, 'somefile'), 'w') as f:
+    nesteddir = os.path.join(testdir, 'test_name')
+    os.makedirs(nesteddir)
+    with open(os.path.join(nesteddir, 'somefile'), 'w') as f:
         f.write('weeeee')
-    source.hoist_single_extracted_folder(testdir)
-    assert os.path.isfile(os.path.join(testing_workdir, 'somefile'))
-    assert not os.path.isdir(testdir)
+    source.hoist_single_extracted_folder(nesteddir)
+    assert os.path.isfile(os.path.join(testdir, 'somefile'))
+    assert not os.path.isdir(nesteddir)
 
 
 def test_append_hash_to_fn(testing_metadata, caplog):