Position: JavaScript-based web scraping
Company: Make it in Ukraine
Category: Data Science
Experience required: 2y
English level: upper
Published: 2020-10-01T00:00:00+03:00

Requirements: 

1.  Chrome’s TamperMonkey or Firefox’ GreaseMonkey or simple bookmarklets to inject code into a web page to then extract its information into a JSON or Avro structure + knowledge about the required JavaScript utility functions.
2.  TypeScript and pure JavaScript (without utility libraries like jQuery) programming skills
3.  Experience developing Browser plugins for Chrome or Firefox.
4.  Knowledge of the available system variables and functions to injected code and browser plugins.
5.  CasperJS or PhantomJS or Slimer.js for headless extraction.
6.  Dumping areas of the DOM to (local) files (requiring special browser permissions or API use) or sending it to backend services.
7.  Inferring relative and absolute coordinates of any DOM element to dump them for later analysis

Optional but important skills:

1.  Experience with programming a purely JavaScript based social bot for one of the top social networks like LinkedIn, Facebook, Xing, Twitter, etc.
2.  Anti JavaScript obfuscation techniques, e.g. decoding binary streams containing JavaScript, overloading the function definition functionality for detecting dynamically generated functions, dumping decoded or dynamically generated functionality, etc.
3.  Anti JavaScript defense techniques to overcome web sites trying to detect web scrapers, injected scripts, ad blockers, etc. I.e. we want to be able to scrape any web page, even if the web site makers did everything to prevent that.
4.  Experience with JavaScript code injection proxies like Zed Attack Proxy (ZAP), Charles Proxy, ProxyStrike, Fiddler, SoapUI.
5.  Experience with cybersecurity and/or social media utility functions, e.g. for better detecting, extracting/scraping elements, dumping the content, overcoming limitations, workarounds for common problems, etc.
6.  Common libraries & services, e.g. for overcoming CAPTCHA checks.