apache/incubator-spark — pull request #539 by ScrapCodes (2014-02-05 10:11:04+00:00)
Title: [WIP] Experiment with java 8 lambdas for our java spark Java API.

Jenkins will not work on this PR, It is only intended for discussion and once we finalize the finer details I promise to please the Jenkins. **There is no reason to have java 8, for this feature (support for java 8) to work**, it is only added for demonstration purposes for discussing the API.

In this PR, I am trying to understand how we can use java 8 lambdas as passable closures to our existing java 6/7 API with minimum or NO impact on the pre-java8 users of the API.

As in this "dirty commit" (as I have given a path to java-home in SparkBuild.scala, which will be convenient for someone who tries this PR out.), Interface IVoidFunction is introduced since in java8 lambdas can only be interfaces and not abstract classes. We internally convert it via implicit to scala-lambda (called Function1 internally.) With this approach user of java 6/7/8 will be able to call the API of foreach in the same way and additionally users of java8 will be able to call the API using java8-lambdas as shown in the JavaAPISuite.

Caveats:
1) With current state our ClosureCleaner does not work with java8-lambdas even if we wrap them, which means when user writes his code in java8 and **also uses lambdas** he will be posed with weird errors by our ClosureCleaner.
2) Our users will be impacted, since the interface IVoidFunction is slightly different as in for instance it has apply method instead of call. This problem can be solved by deprecating the existing way and adding "newer" way as an alternative.

review by @joshrosen and @pwendell