Streamlined crawling and storage process a bit more