added crawler, some method stubs remain todo.