SPARK-2792. Fix reading too much or too little data from each stream in ExternalMap / Sorter

All these changes are from @mridulm's work in #1609, but extracted here to fix this specific issue and make it easier to merge not 1.1. This particular set of changes is to make sure that we read exactly the right range of bytes from each spill file in EAOM: some serializers can write bytes after the last object (e.g. the TC_RESET flag in Java serialization) and that would confuse the previous code into reading it as part of the next batch. There are also improvements to cleanup to make sure files are closed.

In addition to bringing in the changes to ExternalAppendOnlyMap, I also copied them to the corresponding code in ExternalSorter and updated its test suite to test for the same issues.
Are you sure about that? The spark.serializer.objectStreamReset above is set to 0, and that's what causes a TC_RESET after each object written, but the batchSize is about how many objects you write before closing a batch. I definitely saw crashes when I set this to 10 and did not have the fixes in ExternalSorter. There were more when it was 1, so I can also do that. I don't think 0 will work with the code we have written.