The Hadoop Compatibility has been refactored and extended to support the new Java API.

Final PR of FLINK-777. Implements FLINK-611.

Tested for Hadoop 1 and 2. Has been tested on cluster.