added methods to deal with the crawled information