twitter/hadoop-lzo — pull request #74 by rangadi (2013-09-04 08:16:36+00:00)
Title: Handle truncated files and mapper errors

This fixes two issues with DistributedLzoIndexer :+1:

- A truncated file is not an error. We just need to index till the last valid lzo block.
* Fix : Mapper treats EOFException as normal EOF

- When one mapper fails due to hard error, rest of the mappers are also killed. A bad side effect is that DistributedLzoIndexer can't index files when any one of the files can not be indexed. Ideally it should index all the files except the file causing hard error.
* Fix : Each mapper catches IOExceptions and increments a 'SUCCESS' counter when the read was successful (rather than propegating error up resulting whole job being killed). The front end check the counter and detects failure correctly.

currently testing.