Refactored script structure and support single-page
Scraperrr object to structure the interface for later. Scraping and attribute assignment have been moved to methods.
This changes require a custom change to tarantulus: https://github.com/gpolitis/node-tarantula/pull/1
Support single-page Antragsportale
example for single-page scraping for #gvli121
added keywords/tags to motion meta data
added config for lptth131
small refactor, rename to scraperrr.js and removing alternative crawler
read Motion type for lptsh132 example
make scraperrr work on CLI
config and output file configurable by command parameters
updated readme and command line parameters
verbose cl option
updated config json format
The new config format is more complex, but allows
to specify the json-file to export. It's more flexible
and can be extended for improved error handling and
scrape error detection.
updated readme
switch to tarantula 0.0.2