semvec/pypi-packages huggingface dataset:
lich_scrapy_hdfs_pipeline 0.0.1
Auto Generated by os-scrapy-cookiecutter
License: MIT License

# lich_scrapy_hdfs_pipeline

Scrapy project auto generated by [os-scrapy-cookiecutter](https://github.com/cfhamlet/os-scrapy-cookiecutter). 


## Project

This scrapy project is a python package generated by os-scrapy-cookiecutter.

You can run it as normal scrapy project with scrapy commands.

It is also a python package, the scaffolds can be used for formatting, testing, installing.

* lint
  
    ```
    sh scripts/lint.sh
    ```

* test

    ```
    sh scripts/test.sh
    ```

* install

    ```
    python setup.py install
    ```

* example

    You can run example spider:

    ```
    scrapy crawl example
    ```

* files

    ```
    .
    ├── lich_scrapy_hdfs_pipeline # scrapy project          
    │   ├── __init__.py
    │   ├── items.py
    │   ├── middlewares.py
    │   ├── pipelines.py
    │   ├── settings.py
    │   └── spiders
    │       └── __init__.py
    ├── LICENSE # license file
    ├── MANIFEST.in # setup install package file
    ├── pytest.ini # pytest config file
    ├── README.md
    ├── requirements
    │   ├── requirements-lint.txt # requirements of format tools used by scripts/lint.sh and tox
    │   ├── requirements-test.txt # requirements of test used by tox
    │   └── requirements.txt # requirements of this project
    ├── scrapy.cfg # scrapy project entrypoint file
    ├── scripts
    │   ├── test.sh # run test 
    │   └── lint.sh # format the source file
    ├── setup.cfg # setup tool config file
    ├── setup.py # setup tool 
    ├── tests
    │   ├── __init__.py
    │   └── test_scrapy_check.py # unit test with scrapy check
    └── tox.ini # tox config file
    ```

## Install

```
python setup.py install
```

## Usage

```
scrapy -h
```

## Unit Tests

```
sh scripts/test.sh
```

## License

MIT licensed.