twitter/hadoop-lzo — pull request #103 by themodernlife (2014-11-22 16:17:22+00:00)
Title: Remove unnecessary calls to CodecPool.returnCompressor/returnDecompresso...

...r to avoid race conditions

The input/output stream implementations erroneously add the (de)compressors back to the `CodecPool` on close, even though they didn't get the (de)compressors from the pool. The user who creates the (de)compressor is responsibile for doing this and if they both return the decompressor, you will end up with the same instance in the pool twice which leads to a race condition.

This fixes #91 and #94.

There was some concern that this might break some code in the wild.

FWIW, I did a quick search on GitHub to see how people are using this library, and there really wasn't much to speak of outside of forks/hadoop code. The code I did find properly uses `CodecPool` (getting and returning) so this patch wouldn't be an issue. This patch also should work cleanly with any Hadoop setups.

The only way I can see that a user could run into a problem is if they get the decompressor/compressor from the `CodecPool` and then don't return it, in which case they are really using `CodecPool` wrong, which I would hope is not common enough to justify keeping this fix out.

My main motivation is that this makes it possible to use Spark safely with LZO (see #91).

Hope you guys can incorporate it one way or another! Maybe a 0.5.x release (to clearly signal any potential change of behavior)?