cvmfs/cvmfs — pull request #201 by bbockelm (2013-07-08 01:00:56+00:00)
Title: Add runtime checksum verification.

This adds support for runtime checksum verification. This records one 32-bit CRC32 checksum for every 4KB of data. All reads are rounded up to the nearest 4KB block (typically done by FUSE regardless) and the relevant data checksummed before sending data to the client.

The checksumming algorithm is taken from HDFS; the HDFS devs clocked it at 3.7GB/s; I think it's sufficiently performant for the CVMFS use case. I took care to add no locks and minimal allocations for the read() path.

If any checksums fail, then the files are re-downloaded and the read is re-performed -- no error is given to the client.