Moved crawler detection to when the user visits, so crawler info will never even make it into the db