scraping more metadata, handling other file types, not caching landing pages