get list of sns to crawl in the beginning, not including aleady existing
files. saves a lot of time in the beginning.