Moved crawler logic to model from view