Repository: pantsbuild/pants

Issue:
JavacCompileIntegration.test_basic_binary_hermetic is broken under python 2.7
Seen on shard named 'Integration tests - shard 1 (Py2.7 PEX)' in CI cron job of April 14 2019:
```
                   Invalidated 1 target.
07:04:43 21:36       [run]
                     Only executing 1 of 2 total tests in shard 1 of 20
                     F                                          [100%]
                     ==================== FAILURES ====================
                     _ JavacCompileIntegration.test_basic_binary_hermetic _
                     
                     self = <pants_test.backend.jvm.tasks.jvm_compile.javac.test_javac_compile_integration.JavacCompileIntegration testMethod=test_basic_binary_hermetic>
                     
                         def test_basic_binary_hermetic(self):
                           with temporary_dir() as cache_dir:
                             config = {
                               'cache.compile.javac': {'write_to': [cache_dir]},
                               'jvm-platform': {'compiler': 'javac'},
                               'compile.javac': {'execution_strategy': 'hermetic'}
                             }
                         
                             with self.temporary_workdir() as workdir:
                               pants_run = self.run_pants_with_workdir(
                                 ['compile',
                                  'testprojects/src/java/org/pantsbuild/testproject/publish/hello/greet',
                                  ],
                                 workdir, config)
                     >         self.assert_success(pants_run)
                     
                     .pants.d/pyprep/sources/855cc7d8352989ffca502645c0d02c9dc0d3fd48/pants_test/backend/jvm/tasks/jvm_compile/javac/test_javac_compile_integration.py:43: 
                     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                     .pants.d/pyprep/sources/855cc7d8352989ffca502645c0d02c9dc0d3fd48/pants_test/pants_run_integration_test.py:458: in assert_success
                         self.assert_result(pants_run, PANTS_SUCCEEDED_EXIT_CODE, expected=True, msg=msg)
                     .pants.d/pyprep/sources/855cc7d8352989ffca502645c0d02c9dc0d3fd48/pants_test/pants_run_integration_test.py:479: in assert_result
                         assertion(value, pants_run.returncode, error_msg)
                     E   AssertionError: /home/travis/build/pantsbuild/pants/pants --no-pantsrc --pants-workdir=/home/travis/build/pantsbuild/pants/.pants.d/tmp/tmpwNRLLS.pants.d --kill-nailguns --print-exception-stacktrace=True --pants-config-files=/home/travis/build/pantsbuild/pants/.pants.d/tmp/tmpwNRLLS.pants.d/pants.ini compile testprojects/src/java/org/pantsbuild/testproject/publish/hello/greet
                     E   returncode: 1
                     E   stdout:
                     E   	
                     E   	07:04:45 00:00 [main]
                     E   	               (To run a reporting server: ./pants server)
                     E   	07:04:46 00:01   [setup]
                     E   	07:04:46 00:01     [parse]
                     E   	               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> unpack-wheels -> jvm-platform-validate -> deferred-sources -> native-compile -> gen -> resolve -> resources -> compile
                     E   	07:04:46 00:01   [bootstrap]
                     E   	07:04:46 00:01     [substitute-aliased-targets]
                     E   	07:04:46 00:01     [jar-dependency-management]
                     E   	07:04:46 00:01     [bootstrap-jvm-tools]
                     E   	07:04:46 00:01     [provide-tools-jar]
                     E   	07:04:46 00:01   [imports]
                     E   	07:04:46 00:01     [ivy-imports]
                     E   	07:04:46 00:01   [unpack-jars]
                     E   	07:04:46 00:01     [unpack-jars]
                     E   	07:04:46 00:01   [unpack-wheels]
                     E   	07:04:46 00:01     [unpack-wheels]
                     E   	07:04:46 00:01   [jvm-platform-validate]
                     E   	07:04:46 00:01     [jvm-platform-validate]
                     E   	                   Invalidated 1 target.
                     E   	07:04:46 00:01   [deferred-sources]
                     E   	07:04:46 00:01     [deferred-sources]
                     E   	07:04:46 00:01   [native-compile]
                     E   	07:04:46 00:01     [conan-prep]
                     E   	07:04:46 00:01     [conan-fetch]
                     E   	07:04:46 00:01     [c-for-ctypes]
                     E   	07:04:46 00:01     [cpp-for-ctypes]
                     E   	07:04:47 00:02   [gen]
                     E   	07:04:47 00:02     [antlr-java]
                     E   	07:04:47 00:02     [antlr-py]
                     E   	07:04:47 00:02     [jaxb]
                     E   	07:04:47 00:02     [protoc]
                     E   	07:04:47 00:02     [ragel]
                     E   	07:04:47 00:02     [thrift-java]
                     E   	07:04:47 00:02     [thrift-py]
                     E   	07:04:47 00:02     [py-thrift-namespace-clash-check]
                     E   	07:04:47 00:02     [grpcio-prep]
                     E   	07:04:47 00:02     [grpcio-run]
                     E   	07:04:47 00:02     [wire]
                     E   	07:04:47 00:02     [avro-java]
                     E   	07:04:47 00:02     [go-thrift]
                     E   	07:04:47 00:02     [go-protobuf]
                     E   	07:04:47 00:02     [jax-ws]
                     E   	07:04:47 00:02     [scrooge]
                     E   	07:04:47 00:02     [thrifty]
                     E   	07:04:47 00:02   [resolve]
                     E   	07:04:47 00:02     [ivy]
                     E   	07:04:47 00:02     [coursier]
                     E   	07:04:47 00:02     [go]
                     E   	07:04:47 00:02     [scala-js-compile]
                     E   	07:04:47 00:02     [scala-js-link]
                     E   	07:04:47 00:02     [node]
                     E   	07:04:47 00:02   [resources]
                     E   	07:04:47 00:02     [prepare]
                     E   	                   Invalidated 1 target.
                     E   	07:04:47 00:02     [services]
                     E   	07:04:47 00:02   [compile]
                     E   	07:04:47 00:02     [node]
                     E   	07:04:47 00:02     [compile-jvm-prep-command]
                     E   	07:04:47 00:02       [jvm_prep_command]
                     E   	07:04:47 00:02     [compile-prep-command]
                     E   	07:04:47 00:02     [compile]
                     E   	07:04:47 00:02     [rsc]
                     E   	07:04:47 00:02     [zinc]
                     E   	07:04:47 00:02     [javac]
                     E   	                   Invalidated 1 target.
                     E   	07:04:47 00:02       [isolation-java-pool-bootstrap]
                     E   	                   [1/1] Compiling 1 java source in 1 target (testprojects/src/java/org/pantsbuild/testproject/publish/hello/greet:greet).
                     E   	07:04:47 00:02       [compile]
                     E   	                     
                     E   	                   compile(testprojects/src/java/org/pantsbuild/testproject/publish/hello/greet:greet) failed: type check error in class ExecuteProcessRequest: error(s) type checking constructor arguments:
                     E   	                   field 'argv' was invalid: in wrapped constraint TypedCollection(Exactly(unicode)) matching iterable object (u'/usr/lib/jvm/java-8-openjdk-amd64/bin/javac', u'-classpath', u'/home/travis/build/pantsbuild/pants/.pants.d/tmp/tmpwNRLLS.pants.d/compile/javac/8327b2ce6e14/testprojects.src.java.org.pantsbuild.testproject.publish.hello.greet.greet/current/classes:/home/travis/build/pantsbuild/pants/.pants.d/tmp/tmpwNRLLS.pants.d/resources/prepare/0409c64cc4e0/testprojects.src.java.org.pantsbuild.testproject.publish.hello.greet.readme/current', u'-d', u'.', '-encoding', 'UTF-8', '-J-Xmx2g', '-Xlint:deprecation', '-Xlint:empty', '-Xlint:finally', '-Xlint:overrides', '-Xlint:static', '-Xlint:unchecked', '-Xlint:try', u'testprojects/src/java/org/pantsbuild/testproject/publish/hello/greet/Greeting.java'): value '-encoding' (with type 'newstr') must satisfy this type constraint: Exactly(unicode).
                     E   	                   Traceback:
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/backend/jvm/tasks/jvm_compile/execution_graph.py", line 276, in worker
                     E   	                       work()
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/backend/jvm/tasks/jvm_compile/execution_graph.py", line 44, in __call__
                     E   	                       self.fn()
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/backend/jvm/tasks/jvm_compile/jvm_compile.py", line 922, in _default_work_for_vts
                     E   	                       counter)
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/backend/jvm/tasks/jvm_compile/jvm_compile.py", line 512, in _compile_vts
                     E   	                       self._get_plugin_map('scalac', ScalaPlatform.global_instance(), ctx.target),
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/backend/jvm/tasks/jvm_compile/javac/javac_compile.py", line 178, in compile
                     E   	                       self._execute_hermetic_compile(javac_cmd, ctx)
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/backend/jvm/tasks/jvm_compile/javac/javac_compile.py", line 219, in _execute_hermetic_compile
                     E   	                       description='Compiling {} with javac'.format(ctx.target.address.spec),
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/engine/isolated_process.py", line 64, in __new__
                     E   	                       jdk_home=jdk_home,
                     E   	                   
                     E   	                     File "/home/travis/.pex/code/5cbf94785295ac57aba090902a14f9c9fde64dc2/pants/util/objects.py", line 126, in __new__
                     E   	                       .format('\n'.join(type_failure_msgs)))
                     E   	                   
                     E   	FAILURE: Compilation failure: Failed jobs: compile(testprojects/src/java/org/pantsbuild/testproject/publish/hello/greet:greet)
                     E   	
                     E   	
                     E   	               Waiting for background workers to finish.
                     E   	07:04:47 00:02   [complete]
                     E   	               FAILURE
                     E   stderr:
                     -------------- Captured stdout call --------------
                     logs/exceptions.13940.log +++ 
                     logs/exceptions.13940.log --- 
                     logs/exceptions.log +++ 
                     logs/exceptions.log --- 
                     - generated xml file: /home/travis/build/pantsbuild/pants/.pants.d/test/pytest/tests.python.pants_test.backend.jvm.tasks.jvm_compile.javac.javac_compile_integration/junitxml/TEST-tests.python.pants_test.backend.jvm.tasks.jvm_compile.javac.javac_compile_integration.xml -
                     ============ short test summary info =============
                     FAIL tests/python/pants_test/backend/jvm/tasks/jvm_compile/javac/test_javac_compile_integration.py::JavacCompileIntegration::test_basic_binary_hermetic
                     1 failed in 3.88 seconds
```


Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/jvm/tasks/jvm_compile/javac/javac_compile.py b/src/python/pants/backend/jvm/tasks/jvm_compile/javac/javac_compile.py
--- a/src/python/pants/backend/jvm/tasks/jvm_compile/javac/javac_compile.py
+++ b/src/python/pants/backend/jvm/tasks/jvm_compile/javac/javac_compile.py
@@ -212,8 +212,13 @@ def _execute_hermetic_compile(self, cmd, ctx):
       os.path.relpath(f.replace('.java', '.class'), ctx.target.target_base)
       for f in input_snapshot.files if f.endswith('.java')
     )
+
+    # TODO(#6071): Our ExecuteProcessRequest expects a specific string type for arguments,
+    # which py2 doesn't default to. This can be removed when we drop python 2.
+    argv = [text_type(arg) for arg in cmd]
+
     exec_process_request = ExecuteProcessRequest(
-      argv=tuple(cmd),
+      argv=tuple(argv),
       input_files=input_snapshot.directory_digest,
       output_files=output_files,
       description='Compiling {} with javac'.format(ctx.target.address.spec),


Test patch: