making sure there's a trailing / on the domain before crawling.

Signed-off-by: Chris Jones <chris@jonezy.org>