Repository: conda/conda-build

Issue:
`outputs: script:` does not support script arguments, inconsistent with `build: script:`
I'd like to implement the following recipe idea:
```
build:
  script: python build/build.py
outputs:
  - name: tbb
    script: build/build.py --no-rebuild --install-libs
  - name: tbb-devel
    script: build/build.py --no-rebuild --install-devel
```
where `build.py` will handle all the necessary work for all the stages from building to copying for separate parts. Unfortunately, this semantics results in an attempt to copy the entire string as is including command line arguments:
```
Traceback (most recent call last):
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/utils.py", line 152, in _copy_with_shell_fallback
    stderr=subprocess.PIPE, stdout=subprocess.PIPE)
  File "/home/miniconda3/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'cp -a /projects/tbb-svn/tbb/1.0/build/conda_recipes/build/build.py --no-rebuild --install-libs /home/miniconda3/conda-bld/tbb_1510960598212/work/build.py --no-rebuild --install-libs' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/miniconda3/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 388, in main
    execute(sys.argv[1:])
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 379, in execute
    noverify=args.no_verify)
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/api.py", line 185, in build
    need_source_download=need_source_download, config=config, variants=variants)
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1771, in build_tree
    notest=notest,
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1167, in build
    utils.copy_into(os.path.join(meta_dir, output_d['script']), m.config.work_dir)
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/utils.py", line 222, in copy_into
    _copy_with_shell_fallback(src, dst_fn)
  File "/home/miniconda3/lib/python3.5/site-packages/conda_build/utils.py", line 155, in _copy_with_shell_fallback
    raise OSError("Failed to copy {} to {}.  Error was: {}".format(src, dst, e))
OSError: Failed to copy /projects/tbb-svn/tbb/1.0/build/conda_recipes/build/build.py --no-rebuild --install-libs to /home/miniconda3/conda-bld/tbb_1510960598212/work/build.py --no-rebuild --install-libs.  Error was: Command 'cp -a /projects/tbb-svn/tbb/1.0/build/conda_recipes/build/build.py --no-rebuild --install-libs /home/miniconda3/conda-bld/tbb_1510960598212/work/build.py --no-rebuild --install-libs' returned non-zero exit status 1
```
I tried adding e.g. `script_interpreter: python build/build.py --no-rebuild --install-libs` leaving `script: build/build.py` without args: it resulted in an attempt to copy this script from recipe dir instead of source directory. Finally, adding SRC_DIR (`script: {{SRC_DIR}}/build/build.py`) fixed the copying but results in adding excessive last argument to my script_interpreter command. Trying to remove `script:` completely results in not running `script_interpreter:` section.

Any way around this situation doesn't look attractive and consistent. My current work-around is to specify `script: meta.yaml` and `script_interpreter:` section with all the arguments I need while ignoring excessive positional argument inside my build.py...

Please add command line arguments support for `outputs: script:` section



Discussion:
This is an OK feature request, but I can't promise any sort of immediate timeline.  As the docs indicate, "script" in the output section is for script files, which are allowed to be different types of files.  You are trying to shoehorn in your solution, when you could just create a file.  Having an extra file is really not that onerous, so I can't justify prioritizing this highly.  Also, you can have install scripts that utilize the package name to implement different behavior, to limit the overall number of packaging scripts necessary.  See https://github.com/AnacondaRecipes/intel_repack-feedstock/blob/master/recipe/repack.sh for an example.

In the meantime, this might work:

```
build:
  script: {{SRC_DIR}}/build/build.py
outputs:
  - name: tbb
    script: {{SRC_DIR}}/build/build.py
    script_interpreter: python --no-rebuild --install-libs
  - name: tbb-devel
    script: {{SRC_DIR}}/build/build.py
    script_intepreter: python --no-rebuild --install-devel
```

but it might not, since those arguments might get mis-associated with python itself, rather than as args to the script.

The packaging script is expected to come from the recipe, not the source directory, because packaging is a recipe concern, not usually a source package concern.

FWIW, @tomashek requested this before, too.  If this bothers you enough to spend time on a PR, it's very welcome.  I think you'd need to work at https://github.com/conda/conda-build/blob/master/conda_build/build.py#L707 to try to figure out if the script entry was actually a file, and if not, treat it as a shell command.  If you wanted to tie it to the value of script_interpreter, it should write that kind of file, and then use the script_interpreter to execute it.
I understand, it might not be an issue of high importance. Please take it just as a feedback for future evolution. The [principle of the least surprise](http://principles-wiki.net/principles:principle_of_least_surprise) would be enabled if the `output: script` had a different name, e.g. `script_file`.
Few comments regarding the extra files. TBB developers do not like to introduce and maintain additional  files because they internally support a huge table describing each file individually with many properties like where to pack it in Parallel Studio drops, who is the owner and so on. If we would decide to split TBB into multiple separate packages, each for its own clear purpose (e.g. -devel, -doc, -debug, -preview, -examples, -benchmarks, -gcc44, -gcc47, -gcc41, -vc10, -vc12, -vc14), targeting different formats (conda, wheels, rpm, deb), it'd require more than 10 of such proxy files if written in Python.. more if in shell scripts: shell and .bat scripts increase verbosity in the recipe ([win]/[unix] lines), provoke duplication of logic across platforms, which is always error-prone. Thus, I'm trying to avoid using .sh/.bat pairs in my work with conda recipes. While Python, being cross-platform enough, is too bulky to serve as a simple redirect to another script. If you just had bash as a cross-platform shell back-end for conda-build... ohh.
By introducing a single script which knows how to handle all the packaging questions, I also target other package repositories like Fedora's. So I'd ideally like to leave all the conda-specific stuff within single meta.yaml. So far, it works well enough! just requiring this workaround in the build.py to ignore excessive argument.
One suggestion from my recent experience: let `outputs//build/script` override or precede `outputs//script`  and follow regular global `build/script` semantics. I'd not overwrite global value with package-specific though, it would be useful to run global `build/script` as a common pre-build stage for all the package-specific stages (without repeating it).
Making `outputs//test` fully-functional would also be great. Meanwhile I found no way to test imports and use commands as I do in global `test` section
additional issue here is that it seems I cannot control environment through `script_env:` section. TBB makefiles use $(OS) as a way to detect operation system type. Conda-build strips it away for `output/script` execution. I tried both `outputs//script_env` and `outputs//build/script_env` - no effect.


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -721,6 +721,22 @@ def bundle_conda(output, metadata, env, **kw):
     except OSError:
         pass
 
+    # Use script from recipe?
+    script = utils.ensure_list(metadata.get_value('build/script', None))
+    # need to treat top-level stuff specially.  build/script in top-level stuff should not be
+    #     re-run for an output with a similar name to the top-level recipe
+    is_output = 'package:' not in metadata.get_recipe_text()
+    if script and is_output:
+        script = '\n'.join(script)
+
+        suffix = "bat" if utils.on_win else "sh"
+        script_fn = output.get('script') or 'output_script.{}'.format(suffix)
+        with open(os.path.join(metadata.config.work_dir, script_fn), 'a') as f:
+            f.write('\n')
+            f.write(script)
+            f.write('\n')
+        output['script'] = script_fn
+
     if not files or metadata.get_value('build/always_include_files'):
         if output.get('script'):
             with utils.path_prepended(metadata.config.build_prefix):
@@ -735,6 +751,11 @@ def bundle_conda(output, metadata, env, **kw):
             env_output['TOP_PKG_VERSION'] = env['PKG_VERSION']
             env_output['PKG_VERSION'] = metadata.version()
             env_output['PKG_NAME'] = metadata.get_value('package/name')
+            for var in utils.ensure_list(metadata.get_value('build/script_env')):
+                if var not in os.environ:
+                    raise ValueError("env var '{}' specified in script_env, but is not set."
+                                     .format(var))
+                env_output[var] = os.environ[var]
             utils.check_call_env(interpreter.split(' ') +
                         [os.path.join(metadata.config.work_dir, output['script'])],
                                 cwd=metadata.config.work_dir, env=env_output)
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -370,7 +370,7 @@ def meta_vars(meta, config):
         else:
             d[var_name] = value
             warnings.warn(
-                "The environment variable '%s' is being passed through with value %s."
+                "The environment variable '%s' is being passed through with value %s.  "
                 "If you are splitting build and test phases with --no-test, please ensure "
                 "that this value is also set similarly at test time." % (var_name, value),
                 UserWarning


Test patch:
diff --git a/tests/test-recipes/split-packages/_build_script/meta.yaml b/tests/test-recipes/split-packages/_build_script/meta.yaml
new file mode 100644
--- /dev/null
+++ b/tests/test-recipes/split-packages/_build_script/meta.yaml
@@ -0,0 +1,27 @@
+package:
+  name: test_build_script_in_output
+  version: 1.0
+
+# we test to make sure that this script is run only once
+build:
+  script:
+    - set -ex           # [unix]
+    - '[ ! -f "$PREFIX/top_level.txt" ] || exit 1  # [unix]'
+    - echo 'wee' > $PREFIX/top_level.txt  # [unix]
+    - if exist %PREFIX%\top_level.txt exit 1  # [win]
+    - echo 'wee' > %PREFIX%\top_level.txt  # [win]
+
+outputs:
+  - name: test_build_script_in_output
+  - name: test_1
+    build:
+      script_env:
+        # set to test in the test runner code
+        - TEST_FN
+      script:
+        - echo "test" > $PREFIX/$TEST_FN.txt  # [unix]
+        - echo "test" > %PREFIX%\%TEST_FN%.txt  # [win]
+    test:
+      commands:
+        - test -e $PREFIX/test.txt   # [unix]
+        - if not exist %PREFIX%\test.txt exit 1  # [win]
diff --git a/tests/test_subpackages.py b/tests/test_subpackages.py
--- a/tests/test_subpackages.py
+++ b/tests/test_subpackages.py
@@ -277,3 +277,9 @@ def test_subpackage_order_bad(testing_config):
     recipe = os.path.join(subpackage_dir, '_order_bad')
     outputs = api.build(recipe, config=testing_config)
     assert len(outputs) == 2
+
+
+def test_build_script_and_script_env(testing_config):
+    recipe = os.path.join(subpackage_dir, '_build_script')
+    os.environ['TEST_FN'] = 'test'
+    api.build(recipe, config=testing_config)