semvec/pypi-packages huggingface dataset:
scrapy-dynamodb 0.2.3
AWS DynamoDB pipeline for Scrapy

===============
scrapy-dynamodb
===============

scrapy-dynamodb allows you to export your items to a DynamoDB database.

Installation
=============

You can install ``scrapy-dynamodb`` either via the Python Package Index (PyPI)
or from source.

To install using ``pip``,::

    $ pip install scrapy-dynamodb

To install using ``easy_install``,::

    $ easy_install scrapy-dynamodb

Downloading and installing from source
--------------------------------------

Download the latest version of ``scrapy-dynamodb`` from
http://pypi.python.org/pypi/scrapy-dynamodb/

You can install it by doing the following,::

    $ tar xvfz scrapy-dynamodb-0.0.0.tar.gz
    $ cd scrapy-dynamodb-0.0.0
    # python setup.py install # as root

Using the development version
------------------------------

You can clone the git repository by doing the following::

    $ git clone git://github.com/acordiner/scrapy-dynamodb.git
    
Setup
=====================
In ``pipelines.py`` add import statement::   

    from scrapy_dynamodb import DynamoDbPipeline

Using scrapy-dynamodb
=====================

To use ``scrapy-dynamodb`` in your Scrapy project, add
the following line to your ``settings.py``::

    ITEM_PIPELINES = {
        'scraper.pipelines.DynamoDbPipeline': 1,
        # replace scraper with your scrapy project name
    }

    AWS_ACCESS_KEY_ID = '<aws access key id>'
    AWS_SECRET_ACCESS_KEY = '<aws secret access key>'
    DYNAMODB_PIPELINE_REGION_NAME = 'us-east-1'
    DYNAMODB_PIPELINE_TABLE_NAME = 'my_table'
    DYNAMODB_ENDPOINT_URL = '<dynamodb endpoint url>' # optional  (e.g.: http://localhost:8000)

That's it!