Add a sparse chunk index filter

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Add the zero-chunk filter

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Cleanup configuration files

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Refactor the garbage collector.

The GarbageCollector is now an abstract class for which alternative
implementations can be implemented.
Two proposed implementations are a NoneGarbageCollector that does
no garbage collection. This is interessing for research, testing, and
usages where no garbage collection is necessary.
Also a mark&sweep GC would be interessing.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Fix an error in the item counting while dropping pinned entries.

The pinned entries have been dropped correctly, but they have not
been removed from the dirty item count and the total item count.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Work on the statistics maintainance of the block chunk cache.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Fix the emission of the log empty event.

If the replay id has been fully written, we should write the log empty
event.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Add an option to execute the filter chain for a request serially.

The default is still the threadpool-based parallel filter chain
execution.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Adds a basic performance test case to dedupv1

In this test case, two 128 GB files are copied from a different
node using iSCSI.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Remove the system test configuration from internal machines

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Fix the order in which python files are included in dedupv1_util.

The files in /opt/dedupv1/lib should overrule files in the system
directories. This has not been an issue in the previous tightly
controlled installations, but it will become in issue in the future.

Furthermore a ton of trailing whitespaces are removed.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Remove trailing whitespaces from a file.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Removes the sparse filter, the sampling is now in the chunk index.

The problem with the sparse chunk index filter is that various other
components need to know if the chunk index is sparse or not, e.g.
the crash recovery and the garbage collector.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Remove system tests that depend on private files and situation.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>
Block size is 256 unless configured. Remove obsolete configurations.

Signed-off-by: jkaiser <jkaiser@kaiserj-2.zdv.uni-mainz.de>