Fixed numerous small bugs; added hook to crawler