Repository: pantsbuild/pants

Issue:
canonicalize product_types
I am working on a rule to build docker-images. For this I need to isolate binaries from the dependencies.

when you run this command:

    find . -name "*.py" -exec grep -A1 "def product_types(cls):" {} \; | grep  "return \["

you get e.g. 'exe' (cpp rule), 'exec_binary' (go rule), 'pex_archives' (python rules).

This makes it hard to use in a generic way. In my prepare step I don't want to hard code this to specific rules like

    round_manager.require('exec_binary')

and in execute, I'd prefer to not hard-code it again.

    mapping = self.context.products.get_data('exec_binary')

Using mime-types instead of the random string would be nicer, but unfortunately there is no defined mime-type for executables (although you can use 'application/x-executable' or something like it - as long as you define it).


Discussion:
@ensonic : The standard product for this is supposed to be `deployable_archives`, which is exposed by the python and jvm deploy paths. Adding it to the go deployment path would probably be the right approach.
Thanks for the quick reply. Are the types defined somewhere outside of the actual rules? Looks like `deployable_archive` would also need to be added to https://github.com/pantsbuild/pants/blob/master/contrib/cpp/src/python/pants/contrib/cpp/tasks/cpp_binary_create.py
They are not, although we are absolutely moving to using more type information in the new API that is in progress.

If you have time to add it in both places, it would be much appreciated!
PR coming. Looking forward to see the new API. A few comments still:
* deployable_archive is a somewhat uncommon name for an 'executable'.
* maybe consider coming up with a scheme where plugins can define e.g.  'x-my-product' to denote an plugin internal product where the semantics are not public.


Resolving patch:
diff --git a/contrib/cpp/src/python/pants/contrib/cpp/tasks/cpp_binary_create.py b/contrib/cpp/src/python/pants/contrib/cpp/tasks/cpp_binary_create.py
--- a/contrib/cpp/src/python/pants/contrib/cpp/tasks/cpp_binary_create.py
+++ b/contrib/cpp/src/python/pants/contrib/cpp/tasks/cpp_binary_create.py
@@ -22,7 +22,7 @@ def register_options(cls, register):
 
   @classmethod
   def product_types(cls):
-    return ['exe']
+    return ['exe', 'deployable_archives']
 
   @classmethod
   def prepare(cls, options, round_manager):
@@ -39,11 +39,13 @@ def execute(self):
       targets = self.context.targets(self.is_binary)
       with self.invalidated(targets, invalidate_dependents=True) as invalidation_check:
         binary_mapping = self.context.products.get('exe')
+        deployable_archives_mapping = self.context.products.get('deployable_archives')
         for vt in invalidation_check.all_vts:
           binary_path = os.path.join(vt.results_dir, vt.target.name)
           if not vt.valid:
             self._create_binary(vt.target, binary_path)
           binary_mapping.add(vt.target, vt.results_dir).append(binary_path)
+          deployable_archives_mapping.add(vt.target,  os.path.dirname(binary_path)).append(os.path.basename(binary_path))
 
   def _create_binary(self, target, binary_path):
     objects = []
diff --git a/contrib/go/src/python/pants/contrib/go/tasks/go_compile.py b/contrib/go/src/python/pants/contrib/go/tasks/go_compile.py
--- a/contrib/go/src/python/pants/contrib/go/tasks/go_compile.py
+++ b/contrib/go/src/python/pants/contrib/go/tasks/go_compile.py
@@ -30,7 +30,7 @@ def register_options(cls, register):
 
   @classmethod
   def product_types(cls):
-    return ['exec_binary']
+    return ['exec_binary', 'deployable_archives']
 
   def execute(self):
     self.context.products.safe_create_data('exec_binary', lambda: {})
@@ -39,6 +39,8 @@ def execute(self):
                           topological_order=True) as invalidation_check:
       # Maps each local/remote library target to its compiled binary.
       lib_binary_map = {}
+      go_exec_binary = self.context.products.get_data('exec_binary')
+      go_deployable_archive = self.context.products.get('deployable_archives')
       for vt in invalidation_check.all_vts:
         gopath = self.get_gopath(vt.target)
         if not isinstance(vt.target, GoTarget):
@@ -49,7 +51,8 @@ def execute(self):
           self._go_install(vt.target, gopath)
         if self.is_binary(vt.target):
           binary_path = os.path.join(gopath, 'bin', os.path.basename(vt.target.address.spec_path))
-          self.context.products.get_data('exec_binary')[vt.target] = binary_path
+          go_exec_binary[vt.target] = binary_path
+          go_deployable_archive.add(vt.target, os.path.dirname(binary_path)).append(os.path.basename(binary_path))
         else:
           lib_binary_map[vt.target] = os.path.join(gopath, 'pkg', self.goos_goarch,
                                                    vt.target.import_path + '.a')


Test patch: