fix bugs to try to get sharding working
on crawlbot today