adding in basic crawling functionality