Added discoverRegex as an option for the crawler instead of forcing the crawler to use the hardcoded regex