semvec/pypi-packages huggingface dataset:
crawlframe 1.0.3
A Web Crawling and Web Scraping framework (crawlframe) for Python.
License: Apache 2.0
Requires Python: >=3.5.3

CrawlFrame: A Web Crawling framework
==========================
[![PyPI](https://img.shields.io/pypi/v/crawlframe.svg)](https://pypi.org/project/crawlframe/)

##### Behold, the power of CrawlFrame:

``` 
crawlf createapp <appname> -url http://<website>/
crawlf createpro <projectname>
crawlf start app:<name>
crawlf start project
crawlf stop app:<name>
crawlf stop project
crawlf stop <pid>
```

[CrawlFrame](https://github.com/chenxinqun/crawlframe) officially supports python3.5+ .

Installation
------------

To install CrawlFrame, simply use [pip](https://pypi.org/project/crawlframe/):

``` 
$ pip install crawlframe
```


Documentation
-------------
In the construction...


Changed
-------------
[CHANGES.md](https://github.com/chenxinqun/crawlframe/blob/prepack/CHANGES.md)

crawlframe-1.0.3
==========================
下载器并发方式变更



crawlframe-1.0.2
==========================
```
CRAWLFRAME_SURVIVE_SWITCH = True
CRAWLFRAME_SURVIVE_MAX = int
reload spider
```
CRAWLFRAME_SURVIVE_SWITCH = True
CRAWLFRAME_SURVIVE_MAX = int
设置两个参数
发出一定请求数之后, 自动重启爬虫. 避免内存溢出.
修复middle不能加载以及logger不能创建的bug.
BaseLogger改为单例模式.


crawlframe-1.0.1
==========================
``` 
crawlf createapp <appname> -url http://<website>/
crawlf createpro <projectname>
crawlf start app:<name>
crawlf start project
crawlf stop app:<name>
crawlf stop project
crawlf stop <pid>
``` 

实际上以上命令只实现了start 与 stop. 
但是已经能够在项目中使用了.
为了方便自己安装使用, 就先上传到 pypi 与 github 了.