semvec/pypi-packages huggingface dataset:
SoDo 1.0.1
Redis-based  scheduler and Message queue Spider for Scrapy,  Provide more flexible and practical ways for Scrapy
License: MIT License

# SoDo 

## Overview

Redis-based  scheduler and Message queue Spider for Scrapy,  Provide more flexible and practical ways for Scrapy.

For more information [Document](https://ycvbcvfu.github.io/SoDo) [中文文档](https://ycvbcvfu.github.io/SoDo/#/zh-cn/):

## Requirements

- Python 3
- Scrapy >=1.6


## Features

- Distributed 

> Redis-based  scheduler

- Hooks feature

> new  feature for Scrapy projects

- Flexible parameter configuration

> Kafka , Redis client parameter set by yourself.

- Filter interface 

> Built-in BloomFiler and OriginalFilter  




## Installation

```python
pip install -U sodo -i https://pypi.org/simple
``` 


## Contribution guidelines

If you want to contribute to `SoDo`, please see [CONTRIBUTING Doc](https://github.com/ycvbcvfu/SoDo/blob/master/CONTRIBUTING.md)


## For more information

- [HelloWorld Tutorial](https://github.com/ycvbcvfu/SoDo/blob/master/examples/helloworld)




### wechat

![wechat](docs/picture/wechat.JPG)