AG curation crawling framework, factored out of ftp curation tool.
Added m-lab curation tool.