Allow Opts.Delay to override RobotsDelay if longer

If Opts.Delay specifies a longer delay than required by the robots.txt, go with the longer delay value.