don't match url in is_new
remove obsolete scripts
add test and solr related files
remove obsolete tests
remove debug prints
reduce number of tests
only compile scripts
misc script to extract a single download
make less noisy
expect is_new not to match url
run non-db / non-server tests in parallel with compile.t and test_crawler.t