stopped using the ignore numeric last section rule and added in more rate limits instead
fixed dodgy implementation of addUrls