add incremental update of subscription database and crawling of
subscriptions