cvmfs/cvmfs — pull request #106 by reneme (2013-03-07 13:05:47+00:00)
Title: Using an additional thread for concurrent callback handling

This introduces some tweaks in the ConcurrentWorkers template. The callback invocation for finished jobs is now queued and processed in an extra thread. This essentially decouples catalog updates entirely from the worker threads (compression, hashing, chunking).

A quick and dirty benchmark with 30.000 small files (~4kB) shows that the speedup compared to version 2.1.6 is about 25% (on a 4 core ibex node) and 40% with the tunings included in this pull request.

Additionally I refactored the FileProcessor for higher decoupling and information hiding.