Working on moving the deduplication stuff into a client-server system, so I can maintain a large BK tree in memory, rather then having to reload it each time I want to scan something.
Also, moved a bunch of stuff around to make what does what clearer.