Repository: pantsbuild/pants

Issue:
Error in publishing jars with resources after updates
When publishing a jar with resources, the jar does not correctly update when resources are updated. But, the sources jar _does_ update when resources are updated.

Running 1.3.0rc1

## Repro
Have all these in the same directory:

`pants.ini`:
```
[GLOBAL]
pants_version: 1.3.0rc1

[ivy]
ivy_settings: %(pants_supportdir)s/ivysettings.xml

[publish.jar]
dryrun=False
local=~/.m2/repository
prompt=False
changelog=False
named_snapshot=0.1-SNAPSHOT
```

`build-support/ivysettings.xml`:
```
<?xml version="1.0"?>
<ivysettings>
  <settings defaultResolver="chain-repos"/>
  <resolvers>
    <chain name="chain-repos" returnFirst="true">
      <ibiblio name="maven-central" m2compatible="true" descriptor="required"/>
      <filesystem name="local"  m2compatible="true"/>
    </chain>
  </resolvers>
</ivysettings>
```
`BUILD`:
```
scala_library(
    name='MyLibrary',
    sources=['MyClass.scala'],
    dependencies=[':MyResources'],
    provides=artifact(
        org='foo',
        name='my-library',
        repo=repository(
            push_db_basedir='build-support/pushdb'
        ),
    ),
)

resources(
    name='MyResources',
    sources=['hello.txt'],
)
```

`MyClass.scala`:
```
package bar

object MyClass {
  val myVal: Option[Int] = None
}
```

`hello.txt`:
```
hi
```

Run `./pants publish.jar ::` and then:
```
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT.jar hello.txt
hi
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT-sources.jar hello.txt
hi
```

Now update `hello.txt`:
```
bye
```

Run `./pants publish.jar ::` and then:
```
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT.jar hello.txt
hi
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT-sources.jar hello.txt
bye
```


Discussion:
Of a similar repro:
1. Make sure to clear all caches
1. Start as above, publishing the first time
1. Edit `MyResources` to have `sources=[],`
1. Run publish again
```
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT.jar hello.txt        
hi
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT-sources.jar hello.txt
caution: filename not matched:  hello.txt
```

Or another flavor
1. Make sure to clear all caches
1. Start as above, except with `MyResources` not in the dependencies for `MyLibrary`
```
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT.jar hello.txt        
caution: filename not matched:  hello.txt
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT-sources.jar hello.txt
caution: filename not matched:  hello.txt
```
3. Add `MyResources` as a dependencies for `MyLibrary`
3. Publish again
```
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT.jar hello.txt        
caution: filename not matched:  hello.txt
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.1-SNAPSHOT/my-library-0.1-SNAPSHOT-sources.jar hello.txt
hi
```
Thanks for the report @bhmiller !

One thing to try and rule out quickly: do you see the same result when you use different `--named-snapshot` versions for each publish? Colliding/reusing versions are generally problematic (since there is so much caching going on).

Probably unrelated, but note that `--named-snapshot` is not intended to be used to create semver versions (it eagerly fails if you try to give it one).
Changing `pants.ini` to `named_snapshot=0.2-SNAPSHOT` still exhibits the bad behavior:
```
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.2-SNAPSHOT/my-library-0.2-SNAPSHOT.jar hello.txt
caution: filename not matched:  hello.txt
$ unzip -q -c  ~/.m2/repository/foo/my-library/0.2-SNAPSHOT/my-library-0.2-SNAPSHOT-sources.jar hello.txt
hi
```
(starting point for this being my last comment's 2nd case)
On previous comment, that was from the first time publishing with the updated `named_snapshot` setting in `pants.ini`
Will look at this one tonight.
I repro... nice find. Looks like an invalidation bug in the `publish.jar` task. As a workaround, should be able to run `./pants clean-all` before publishing.

But also occurs in the `1.2.1` release, so I'm going to remove this from the `1.3.x` release.
thanks for the update
We should be able to target to `1.3.1` though, as soon as we get `1.3.0` out the door.
thoughts on tagging this to the 1.3.1 milestone now that 1.3.0 is done? thanks.
Thanks, done.

Resolving patch:
diff --git a/src/python/pants/backend/jvm/tasks/jar_create.py b/src/python/pants/backend/jvm/tasks/jar_create.py
--- a/src/python/pants/backend/jvm/tasks/jar_create.py
+++ b/src/python/pants/backend/jvm/tasks/jar_create.py
@@ -62,7 +62,14 @@ def cache_target_dirs(self):
     return True
 
   def execute(self):
-    with self.invalidated(self.context.targets(is_jvm_library)) as invalidation_check:
+    # NB: Invalidating dependents transitively is more than is strictly necessary, but
+    # we know that JarBuilderTask touches (at least) the direct dependencies of targets (in
+    # the case of resources). One of these tasks could implement an FingerprintStrategy that
+    # would attempt to hash the relevant dependencies, but that is really error prone, and
+    # this task is more than fast enough to re-run (JarTool "copies" pre-zipped data from input
+    # zip/jar files).
+    with self.invalidated(self.context.targets(is_jvm_library),
+                          invalidate_dependents=True) as invalidation_check:
       with self.context.new_workunit(name='jar-create', labels=[WorkUnitLabel.MULTITOOL]):
         jar_mapping = self.context.products.get('jars')
 


Test patch:
diff --git a/tests/python/pants_test/backend/jvm/tasks/test_jar_publish_integration.py b/tests/python/pants_test/backend/jvm/tasks/test_jar_publish_integration.py
--- a/tests/python/pants_test/backend/jvm/tasks/test_jar_publish_integration.py
+++ b/tests/python/pants_test/backend/jvm/tasks/test_jar_publish_integration.py
@@ -9,7 +9,7 @@
 import re
 
 from pants.base.build_environment import get_buildroot
-from pants.util.contextutil import temporary_dir
+from pants.util.contextutil import open_zip, temporary_dir
 from pants.util.dirutil import safe_rmtree
 from pants_test.pants_run_integration_test import PantsRunIntegrationTest
 
@@ -217,6 +217,40 @@ def test_override_via_address(self):
       extra_options=['--override={}=1.2.3'.format(target)],
       assert_publish_config_contents=True)
 
+  def test_invalidate_resources(self):
+    """Tests that resource changes invalidate publishes."""
+    source_root = 'testprojects/src/java'
+    target_relative_to_sourceroot = 'org/pantsbuild/testproject/publish/hello/greet'
+    target = os.path.join(source_root, target_relative_to_sourceroot)
+    resource_relative_to_sourceroot = os.path.join(target_relative_to_sourceroot, 'TEMP.txt')
+    resource = os.path.join(source_root, resource_relative_to_sourceroot)
+
+    with self.temporary_workdir() as workdir:
+      def publish(resource_content):
+        with temporary_dir() as publish_dir:
+          with self.temporary_file_content(resource, resource_content):
+            # Validate that the target depends on the relevant resource.
+            self.assertIn(resource, self.run_pants(['filedeps', target]).stdout_data)
+
+            pants_run = self.run_pants_with_workdir(['publish.jar',
+                                                     '--local={}'.format(publish_dir),
+                                                     '--named-snapshot=X',
+                                                     '--no-dryrun',
+                                                     target
+                                                    ],
+                                                    workdir=workdir)
+            self.assert_success(pants_run)
+          # Validate that the content in the resulting jar matches.
+          jar = os.path.join(publish_dir,
+                             'org/pantsbuild/testproject/publish/hello-greet/X/hello-greet-X.jar')
+          with open_zip(jar, mode='r') as j:
+            with j.open(resource_relative_to_sourceroot) as jar_entry:
+              self.assertEquals(resource_content, jar_entry.read())
+
+      # Publish the same target twice with different resource content.
+      publish('one')
+      publish('two')
+
   def publish_test(self, target, artifacts, pushdb_files, extra_options=None, extra_config=None,
                    extra_env=None, success_expected=True, assert_publish_config_contents=False):
     """Tests that publishing the given target results in the expected output.