NOTE: big change! Need to delete config files. The crawler will cache hashes and merge duplicated queries.