[SPARK-2532] Consolidated shuffle fixes

Status of the PR
- [X] Cherry pick and merge changes from internal branch to spark master
- [X] Remove WIP comments and 2G branch references.
- [X] Tests for BlockObjectWriter
- [x] Tests for ExternalAppendOnlyMap
- [x] Tests for ShuffleBlockManager



The only reason to use eq/ne was to avoid the call to equals/canEquals/etc.
Since spark ships with assertions turned on, this ends up being expensive since some of the codepaths are very frequent : and all changes in the 2G fix branch uses eq/ne.
Not sure how relevant it is to this patch though ... I can revert to ==/!= if it is a problem !