GitHub repository: steven-mi/NewsPipe
Primary language: Python
Created: 2020-08-30T18:20:47Z
Description: This repository contains the complete pipeline for crawling online newspaper article. The articles are stored to MongoDB. The whole pipeline is dockerized, thus the user does not need to worry about dependencies. Additionally, docker-compose is available to increase the useability for the user. Local, Sequential and Celery Executor are supported.