Yes we will take notice of the robots.txt file and crawl any sitemap mentioned in the file.