semvec/pypi-packages huggingface dataset:
pydq 0.1.20
Python library for managing and persisting named Python Queues with a variety of backend choices

# pydq
A python library for managing and persisting named Python Queues

### Installation
```bash
pip install pydq
```
Some backends may require extra packages:
```bash
pip install pydq[aws]
```

### Basic Usage
```python
with dq_provider('name') as dq:
    # Query the last 10 entries with qid='item_id'
    dq(qid='item_id', start_time=datetime(1, 1, 1), end_time=datetime.utcnow(), limit=10)
    item = dq.get()
    # queue items should be dictionaries with 3 keys: qid, ts, and val
    dq.put({'qid': 'item_id', 'ts': '2019-09-11T03:23:54Z', 'val': 'foo'})
```
note: specifying `qid` in the query is optional, if omitted all items between the start and end time are returned
### Supported Backends:
#### Local Storage
* SQLite:
```python
from pydq.local import SQLite as dq_provider
```

#### Cloud Storage
* DynamoDB
```python
from pydq.cloud import DynamoDB as dq_provider
```
note: cloud storage is created lazily; the first call to one of these providers will create the resources in the cloud. 

#### Additional API
Some queue methods accept a `forget` argument that will tell the queue object whether the operation should be persisted. This always defaults to `False`.
```python
dq.get(forget=True)  # the returned item will be deleted from persistence
dq.put(item, forget=True)    # item will not be persisted
dq.get_all(forget=True)  # Easy way to delete all items in a queue
dq.put_all(items, forget=False)
dq.get_to_dataframe()    # install with pandas extra. Returned items are not deleted
dq.put_from_dataframe(df)    # items will be persisted
dq.clear()    # another way to delete all items in a queue
```

#### Configuration queues
dq is great for managing simple JSON configuration too. And we get change tracking for free!

```python
from pydq.helpers import DqConfig
from pydq.local import SQLite
with DqConfig(name='ApplicationConfig', dq_provider=SQLite) as config:
    app_id = config['app_id']
    last_start = config['last_start']
    config['app_status'] = '#1best'
```