SPARK-964, Java 8 API Support.

This patch adds a few methods to java API such that it is possible to pass lambdas instead
of Anonymous classes and also java 6/7 api users can use the same apis by passing anonymous classes.
To achieve this a few older API methods are removed and replaced with their ToPair/ToDouble versions.

1) all anonymous classes extending scala Function is replaced by interfaces.

2) Adds optional to run java 8 tests

Please refer to PR comments for more details.