semvec/pypi-packages huggingface dataset:
os-scrapy-aio-resolver 0.0.3
Auto Generated by os-scrapy-cookiecutter
License: MIT License
Requires Python: >=3.6

# os-scrapy-aio-resolver

[![Build Status](https://www.travis-ci.org/cfhamlet/os-scrapy-aio-resolver.svg?branch=master)](https://www.travis-ci.org/cfhamlet/os-scrapy-aio-resolver)
[![codecov](https://codecov.io/gh/cfhamlet/os-scrapy-aio-resolver/branch/master/graph/badge.svg)](https://codecov.io/gh/cfhamlet/os-scrapy-aio-resolver)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/os-scrapy-aio-resolver.svg)](https://pypi.python.org/pypi/os-scrapy-aio-resolver)
[![PyPI](https://img.shields.io/pypi/v/os-scrapy-aio-resolver.svg)](https://pypi.python.org/pypi/os-scrapy-aio-resolver)

This project provide a async DNS resolver.

## Install

```
pip install os-scrapy-aio-resolver
```

You can run example spider directly in the project root path

```
scrapy crawl example
```

## Usage

### Settings

* enable DNS resolver in the project settings file:

    ```
    DNS_RESOLVER = "os_scrapy_aio_resolver.AsyncResolver"
    ```

* [use asyncio reactor](https://docs.scrapy.org/en/latest/topics/asyncio.html)

    ```
    TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"
    ```

    or you can use os-scrapy command line to start crawling with ``-r asyncio``

    ```
    os-scrapy crawl example -r asyncio
    ```

* enable/disable DNS cache, set cache size

    ```
    DNSCACHE_ENABLED = True
    DNSCACHE_SIZE = 10000
    ```

## Unit Tests

```
sh scripts/test.sh
```

## License

MIT licensed.