feat(js): correct robots and sitemap crawl