Full code review

I suppose you used this for unittesting. In the end you can remove this or, better, put it in the unittests directory together with its unittests. This will assure that , if someone modifies the code in future, it will still run on ClueWeb.