The Hadoop Compatibility has been refactored and extended to support the new Java API.

Final PR of FLINK-777. Implements FLINK-611.

Tested for Hadoop 1 and 2. Has been tested on cluster. 
we should mark the `recordReader` as transient as well. Even though we do custom serialization, it is helpful as a flag when reading the code (similar to key and value, the transient flag has actually no meaning here, since we do custom ser)