QFS status dashboard now sends HTTP Content-Length header for the /files
and /charts resources (fixes 'infinite' wait when behind a proxy such as
NGINX)
May now specify the listening ip address for the QFS status dashboard
through the webServer.host parameter in "server.conf". Defaults to
0.0.0.0
Client library: fix compare chunk replicas to skip chunk with no replicas.
Merge pull request #34 from coderfi/master

QFS status dashboard now sends HTTP Content-Length header for the /files
Chunk server: add chunkServer.rsReader.maxRecoverChunkSize config option to facilitate sparse files recovery debugging.
Qfs fuse: allow to build qfs_fuse on macos with osxfuse port.
Chunk server: fix end of chunk flag set in chunk recovery code for the case where stripe size isn't multiple of check sum block size.
Meta server: implement "force chunk replication" RPC in order to facilitate [sparse files replication] debugging. The RPC allows to schedule chunk replication or recovery on a particular server, if specified, or the server chosen by the meta server according to chunk placement constrains. The chunk replication will be started even in the case when the meta server is in "recovery" mode.
Client library: add force chunk replication RPC id.
Tools: allow to specify RPC fields with -F command line option in QFS admin tool. Implement force chunk replication RPC with admin tool.
Client library: fix recovery stripe recovery with sparse file by using the computed recovery size, instead of read size, as the computed value takes into the account the end of RS block / hole start position. This only has effect when the recovery stripe become available while / before recovery starts, and some other stripes (chunks) becomes unavailable. Without this change the recovery stops only at the max chunk size, instead at the hole boundary. The boundary only knows with stripe size precision, in the case when the chunk that contains the partial stripe with the hole boundary isn't available (lost).
Client library: handle possible loss of chunk block end / hole start position with sparse stripe files, resulting in possibly larger size of the recovered chunk files, by allowing chunks larger by no more than stripe size, with the zero filled trailing part that spans past the hole boundary. It is possible that the original chunk replica with the stripe containing the hole boundary re-appeared, and the chunk replica containing subsequent stripe was recovered without exact knowledge of the size of the original stripe.
Client library: fix CompareChunkReplicas() return code to ensure that if reflects the last error, if any.
Meta server: change panic on invalid chunk debug logic, to panic only in the case if the chunk belong to logically non empty file, and check permissions when reporting invalid chunks in the case if authentication is enabled.
Client library: RS recovery: pass chunk size from the reader to RS striper in order to allow better detection of RS block end with sparse files, when the recovery size is less than stripe size.If the end of the data stripe is stripe aligned, see if the end is in preceding stripes. The stripe with the end that isn't stripe aligned is where the end is. If the recovery size is less than stripe size, the read of this stripe might not not return less that recovery size. Ensure that the data stripe prior to the recovered data stripe is *not* selected as bad stripe, as it might define the hole position in the case when the file is sparse.
Test scripts: create test script to test recovery of partial RS blocks with sparse files.
Kfsio library: implement IOBuffer::CopyIn method with position between "used"/"consumed" and available space, to avoid searching for the boundary, by traversing the scatter-gatherer buffers linked list..
Client library: use IOBuffer::CopyIn() method with position argument, in order to get rid of O^2 linked list traversal. The traversal is a major cpu hog with chunk server where io buffer size is 4K and typical recovery size is 1MB, or 256 buffers / linked list entries.
Kfsio library: change IOBuffer::ZeroFill() method to use available space, if any, in order to make the behavior consistent with other IOBuffer methods. Make IOBuffer::CopyIn(pos) method work with pos set to end of the buffer list to make the behavior consistent with other IOBuffer methods.
Client library: fix sparse files holes zero fill by using ZeroFillSpaceAvailable() instead of ZeroFill(), as the later wasn't using the available space prior to the last change, and it will extend the io buffer, if needed. Fix space available handling in the kfs protocol state machine response handler in the case where only part of the available space used: ensure that the unused portion of the available space still remains in the buffer, by moving this portion of the space back to the buffer the response content.
Kfsio library: optimize available space search for empty buffers (buffers zero "consumed" space).