Headless browser used for rendering HTML for web applications to support web crawlers