ScrapeServer now writes to directory incremenetally

This solves the problem that qdel sends a sigint to the job, which is uncatchable by the script. Now the directory.json file is actually written. Note: links.json can't be written incrementally so it is not utilized on the CLSP grid (referal links are still used, they just don't persist between runs).