Repository: conda/conda-build

Issue:
Duplicated test imports (and commands?)
During testing, seems that conda build repeats the test imports (and maybe the commands?) multiple times.

### Actual Behavior

```
$ cat conda.recipe/meta.yaml
package:
  name: test
  version: 0.1

source:
  path: ..

build:
  noarch: python
  script: python setup.py install --single-version-externally-managed --record=record.txt

requirements:
  build:
    - python
    - setuptools
  run:
    - python
test:
  imports:
    - test
  commands:
    - echo 1
    - echo 2
    - echo 3
    - false

$ conda build conda.recipe
[...]
TEST START: /Users/[...]/conda-bld/noarch/test-0.1-py_0.tar.bz2
[...]
Executing transaction: ...working... done
import: 'test'
import: 'test'
import: 'test'
+ echo 1
1
+ echo 2
2
+ echo 3
3
+ false
Tests failed for test-0.1-py_0.tar.bz2 - moving package to /Users/[...]/conda-bld/broken
WARNING:conda_build.build:Tests failed for test-0.1-py_0.tar.bz2 - moving package to /Users/[...]/conda-bld/broken
removing: test-0.1-py_0.tar.bz2
TESTS FAILED: test-0.1-py_0.tar.bz2

$ cat [...]/conda-bld/test_1524047912513/test_tmp/run_test.py
print("import: 'test'")
import test

print("import: 'test'")
import test

print("import: 'test'")
import test


$ cat [...]/conda-bld/test_1524047912513/test_tmp/run_test.sh


set -ex



echo 1
echo 2
echo 3
false
exit 0


set -ex



echo 1
echo 2
echo 3
false
exit 0


set -ex



echo 1
echo 2
echo 3
false
exit 0
```

### Expected Behavior

No repetition of imports or commands.

### Steps to Reproduce

Can run the commands above using the sample python package in [test.zip](https://github.com/conda/conda-build/files/1923666/test.zip)



##### `Output of conda info`

```
$ conda info

     active environment : dsdev
    active env location : /Users/[...]/envs/dsdev
            shell level : 1
       user config file : /Users/[...]/.condarc
 populated config files :
          conda version : 4.5.1
    conda-build version : 3.8.1
         python version : 3.6.4.final.0
       base environment : /Users/[...]  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/osx-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /Users/[...]/pkgs
                          /Users/[...]/.conda/pkgs
       envs directories : /Users/[...]/envs
                          /Users/[...]/.conda/envs
               platform : osx-64
             user-agent : conda/4.5.1 requests/2.18.4 CPython/3.6.4 Darwin/16.7.0 OSX/10.12.6
                UID:GID : 501:20
             netrc file : None
           offline mode : False
```




Discussion:
I ran across this issue awhile ago with run_test.py, but I assume the same logic is responsible for run_test.sh. What I discovered was:

The only way to generate a run_test.py like that is repeated calls to create_test.py/create_py_files.
The three calls seems to be coming from
1. build.py/testL1764
```python
copy_test_source_files(metadata, metadata.config.test_dir)
```
which calls `create_all_test_files()`
2. build.py/testL1767
```python
_, pl_files, py_files, r_files, lua_files, shell_files = \
    create_all_test_files(metadata, existing_test_dir=metadata.config.test_dir)
```
3. build.py/create_info_filesL617.  I'm not 100% certain if this the third invocation, but it's the only other place I can trace a call to `create_all_test_files()` back to.
```python
if m.config.copy_test_source_files:
    copy_test_source_files(m, join(m, join(m.config.info_dir, 'test'))
```

I attempted to fix in two ways:
1. Making the file mode 'w' instead of 'a'.
2. Adding checks to make sure that `create_all_test_files()` only runs once.

My changes seemed to break many tests though.  I haven't had time pick this up again.
Thanks @groutr for doing that legwork.  That stuff really needs a refactor.  It is an unhappy amalgamation of the old way, where tests always relied on having the original recipe (and source, too, I think), and the new way, where the tests can be baked into packages and no longer depend on embedding the recipe into the package.
This should be fixed soon as it triples CI time for testing when using conda build. Thanks for reporting and working on this!
I don't see this on master anymore.  What conda-build version are you running?  Have you tried 3.10.2?  3.10.3 or 3.11.0 will be out soon, and if this is still a problem, I'd like to fix it.
I used 3.10.2
Same on master.
yes, bizarre.  Somehow I see it now, but did not before.  I'll look into it for the next release.
it seems like it is sufficient to remove the create_all_test_files in the copy_test_source_files call.  Thanks @groutr for the analysis.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -360,8 +360,14 @@ def copy_license(m):
 
 
 def copy_test_source_files(m, destination):
-    create_all_test_files(m, destination)
+    # if we include the recipe, we create the test files there.  Otherwise, create them here.
+    if not(m.config.include_recipe and m.include_recipe()):
+        create_all_test_files(m, destination)
     test_deps = m.meta.get('test', {}).get('requires', [])
+    try:
+        os.makedirs(destination)
+    except:
+        pass
     if test_deps:
         with open(os.path.join(destination, 'test_time_dependencies.json'), 'w') as f:
             json.dump(test_deps, f)


Test patch: