basho/bitcask — pull request #66 by slfritchie (2012-10-27 02:46:48+00:00)
Title: Fix log spam introduced by branch 'gh62-badrecord-mstate'

So, I worried a bit about the possibility of spamminess of
the changes made by branch 'gh62-badrecord-mstate' at the time
I wrote it, but not enough. In a pull-the-power-cord failure,
the next Riak start included tens of thousands of error messages.
See example below.

Fix: Introduce a cutoff of 20 CRC errors when folding over any
single file.

( echo "line-count path bytes-skipped" ; grep 'skip' ALL.console.log | awk '{print $11, $15}' | uniq -c | sort -k 2 | sed 's/^ *//' ) | sed 's/ /, /g'
42918, /var/lib/riak/bitcask/1019054852592055355075616182186939527377281089536/3.bitcask.data, 14
1, /var/lib/riak/bitcask/1183188337253240741397319068673631468061296951296/3.bitcask.data, 1049176
34757, /var/lib/riak/bitcask/1218869529570889738423776217909868846470865616896/3.bitcask.data, 14
23, /var/lib/riak/bitcask/1247414483425008936044941937298858749198520549376/3.bitcask.data, 14
1, /var/lib/riak/bitcask/1283095675742657933071399086535096127608089214976/3.bitcask.data, 1049177
23444, /var/lib/riak/bitcask/1283095675742657933071399086535096127608089214976/3.bitcask.data, 14
1, /var/lib/riak/bitcask/1290231914206187732476690516382343603290002948096/3.bitcask.data, 1049177
36198, /var/lib/riak/bitcask/1290231914206187732476690516382343603290002948096/3.bitcask.data, 14
292, /var/lib/riak/bitcask/248341098530837019304141758684212153730597912576/3.bitcask.data, 14
258, /var/lib/riak/bitcask/398202106264962806815261785476409143050786308096/3.bitcask.data, 14
165, /var/lib/riak/bitcask/419610821655552205031136075018151570096527507456/3.bitcask.data, 14
156, /var/lib/riak/bitcask/433883298582611803841718934712646521460354973696/3.bitcask.data, 14
1, /var/lib/riak/bitcask/533790637072028995515798952574111181007147237376/3.bitcask.data, 1049176
3954, /var/lib/riak/bitcask/533790637072028995515798952574111181007147237376/3.bitcask.data, 14
1, /var/lib/riak/bitcask/626561737097916387784587540588328364872025767936/3.bitcask.data, 1049174
77531, /var/lib/riak/bitcask/62798898479062234766564582655777786000840851456/3.bitcask.data, 14
41394, /var/lib/riak/bitcask/783558983295571974700998997227772829874127896576/3.bitcask.data, 14