elasticsearch/elasticsearch — pull request #8399 by rmuir (2014-11-08 04:23:12+00:00)
Title: Internal: harden recovery for old segments

When a lucene 4.8+ file is transferred, Store returns a VerifyingIndexOutput
that verifies both the CRC32 integrity and the length of the file.

However, for older files, problems can make it to the lucene level. This is not great
since older lucene files aren't especially strong as far as detecting issues here.

For example, if a network transfer is closed on the remote side, we might write a
truncated or corrupted file... which old lucene formats may or may not detect.

The idea here is to verify old files with their legacy Adler32 checksum, plus expected
length. If they don't have an Adler32 (segments_N, jurassic elasticsearch?, its optional
as far as the protocol goes), then at least check the length.

We could improve it for segments_N, its had an embedded CRC32 forever in lucene, but this
gets trickier. Long term, we should also try to also improve tests around here, especially
backwards compat testing, we should test that detected corruptions are handled properly.

I added unit tests for the two cases covered here. PR is against 1.x since thats what i'm working with,
but I can forward-port to master once we figure this out.