Repository: google/protobuf-gradle-plugin

Issue #331: The plugin does not detect proto file was changed when running build for a second time
Created: 2019-08-28T17:57:43Z
Use the gradle plugin to generate code, then change the proto file, then build again.
As reproduced in https://sponge.corp.google.com/target?id=ff244382-5595-48fd-b556-4b2694aaf5d4&target=grpc%2Fjava%2Fmaster%2Fpresubmit%2Fandroid&searchFor=

Duplicate issue #464: Generating protobuf classes without cleaning the generated classes first interacts poorly with gradle caching when java_multiple_files is true or multiple proto files are used
Created: 2021-01-24T18:32:24Z
Issue
If you do something that removes files from the generated Java output directory, like removing a message definition, but you don't clean first, gradle will cache the not-removed-but-intended-to-be-removed .java file along with your new definitions. From that point forward, cleaning will not remove the file, even if you regenerate the proto .java files, because the result is cached.
In other words, the way the plugin defines outputs interacts poorly with the cache.
Solutions

Don't use the directory as the outputs for a cacheable task, use the exact files that are expected to be generated. Otherwise you'll cache files that should be deleted, if the user did not clean before hand.
Correctly delete the contents of the target directory. Sometimes messages are removed from a protobuf file during development, for example, so it should be cleaned.
Error on option java_multiple_files = true;.

Reproduction

Add org.gradle.caching=true to your gradle.properties.
Create a .proto file that looks like this.

option java_multiple_files = true;
message ShouldExist {
  int32 field = 1;
}

message ShouldBeRemoved {
  int32 field = 1;
}


Run generateProto. Observe you get: ShouldExist.java and ShouldBeRemoved.java.
Remove ShouldBeRemoved from your .proto file.
Run generateProto. Observe you still have ShouldExist.java and ShouldBeRemoved.java. Observe this task now caches the whole generated files directory, including ShouldBeRemoved.java.
Run clean. Observe you have no generated files.
Run generateProto. Observe you still have ShouldExist.java and ShouldBeRemoved.java, even though ShouldBeRemoved.java should be removed. Why? It was cached!
Add a comment or any other cache-busting change to your .proto file.
Run clean
Run generateProto. Observe you only have ShouldExist.java.

Notes
This problem also exists if you use multiple proto files and delete one. It's not just limited to option java_multiple_files. No matter what, you have to correctly register outputs for inputs if you want gradle caching to work correctly.

Closing comment (2022-11-01T16:47:39Z): Closing since this is a dup of #331 and it is confusing to have both, even if this is more caching-directed.
We know how to reproduce this issue, and I have worked some on a fix. It isn't a hard fix, but it is non-trivial and requires attention to detail.
I had worked on fixing this but was delayed and didn't want to delay the 0.9.0 release, and now there's fall-out from the 0.9.0 release. Honestly, Bazel goes out of its way to prevent this sort of issue while Gradle doesn't have a restrictive runtime environment so it is easy to do things wrong and not notice. That's not surprising as the build cache was a later addition to Gradle. Part of the trouble, though, is also this issue is super-old and Gradle has advanced a lot since the plugin was initially written.
This issue is caused by generatedFilesBaseDir (and its predecessor generatedFileDir) which existed at least (46935b3) in 2013, before this project was adopted by Google in 2015. And that predates the Gradle build cache.