Working prototype of one-way (half-duplex) planner => agent communication
One file for declaring AMQP exchanges and queues
Basic communication fetcher agent => planner.fetched
Fixed tests
Basic torso of CrawlerAgent
Basic planner => fetcher => planner => crawler => planner pipeline ;-)
More smart raw_docs caching algorithm
Engaged threadpool - https://github.com/meh/ruby-thread
Created DataSource model to keep info about (surprisingly) data source - website, host, etc
Disabled DNS monkey patch not working with latest
Added missing source
Cosmetics
Fixed adding next crawling links from fetched and crawled docs
Almost properly working enqueuing alg...
Preparing for engaging scheduler
Engaged URL queuing scheduler
Not process any queued URL again - temporary
Added Alexa Rank data => we need some food for "Domainer"
Updated Gem date
Merge pull request #47 from korczis/feat-paas

Few PaaS Steps