semvec/pypi-packages huggingface dataset:
envbox 2.0.1
Detect environment type and work within.
Requires Python: >=3.10
Keywords: backup, webinars

# envbox

https://github.com/idlesign/envbox

[![PyPI - Version](https://img.shields.io/pypi/v/envbox)](https://pypi.python.org/pypi/envbox)
[![License](https://img.shields.io/pypi/l/envbox)](https://pypi.python.org/pypi/envbox)
[![Coverage](https://img.shields.io/coverallsCoverage/github/idlesign/envbox)](https://coveralls.io/r/idlesign/envbox)
[![Docs](https://img.shields.io/readthedocs/envbox)](https://envbox.readthedocs.io/)


## Description

*Detect environment type and work within.*

### Features

* Environment type detection (extendable system);
* Support for `.env` files;
* Convenient `os.environ` proxying (with optional values casting into Python natives);
* Automatic submodule-for-environment import tool;
* Cosy per-thread settings container with environment var support;
* CLI for environment probing.


## Code sample

```python
from envbox import get_environment

# Detect current environment type
# and get its object.
#
# Default detection sources:
# 1. ``PYTHON_ENV`` env variable
# 2. ``environment`` file contents
#
# By default this function will also try to read env variables from .env files.
env = get_environment()

env.name
# >> development

env.is_production
# >> False

env.get('HOME')  # The same as env['HOME'] and env.HOME
# >> /home/idle/

env.getmany('PYTHON')
# {'UNBUFFERED': '1', 'IOENCODING': 'UTF-8', 'PATH': ...}
```

## .env files

Now you may want to put your environment vars into `.env` files
(e.g.: `.env`, `.env.development` `.env.production`) to be read by `envbox`:

```dotenv
MY_VAR_1=value1
HOME=/home/other/

# comments are ignored, just as lines without definitions

# mathing quotes (" and ') are stripped
MY_QUOTED="some quoted "

# ${VARNAME} will be replaced by value from env (if available)
MY_VAR_2="${MY_QUOTED}"

# multiline with dangling quotes
MULTI_1="
line1
line2
"

# multiline classic
MULTI_2="line1
line2
line3"

# multiline as one line
MULTI_3="one\ntwo"
```
Read the docs for more examples.


## CLI

```shell
$ envbox probe
# >> Detected environment type: development (Development)

$ envbox show
# >> [...]
# >> SHELL = /bin/bash
# >> [...]
```

!!! note
    `envbox` CLI requires `click` package available.


## Documentation

https://envbox.readthedocs.io/
datatia 0.4.5
Opinionated declarative utility library for writing PyTorch dataset classes
Requires Python: >=3.9

# What is it?
Opinionated declarative utility library for writing dataset classes. Intended for small pytorch experiments.

# Rationale
Pytorch dataset loading often involves certain common tasks:
- Load tensors or values from a filelist
- Truncate sequence/spatial dims to a maximum length
- Drop items that don't satisfy particular requirements
- Pad sequence/spatial dims to a multiple of a number or a maximum per-batch length
- Pad sequence/spatial dims in groups across multiple data fields in a batch
- Or (on training datasets only) randomly subsample sequence/spatial dims to meet a maximum length constraint, and add a "length" field for the pre-padding lengths
- Apply data augmentations

Implementing these tasks is often highly repetitive and error prone.

Dataset loading code can be further simplified by making certain assumptions:
- All data to be loaded takes the form of either a literal or a single file tensor which can be loaded from disk.
- Each dataset class takes only one filelist.
- Filelists contain only paths to tensors or python literals (such as class IDs).

# Example usage
We use an example a 3-column dataset specified as a filelist:
```yaml
# filelist.txt
test/test_files/tensor1_0.pt|test/test_files/tensor2_0.pt|0
test/test_files/tensor1_1.pt|test/test_files/tensor2_1.pt|1
test/test_files/tensor1_2.pt|test/test_files/tensor2_2.pt|2
```

When creating a `dt.Dataset` we specify names and datatypes for the columns in order.
```python
import datatia as dt
dataset = dt.Dataset(filelist='filelist.txt',
    field_specs=[
        dt.FieldSpec(name='tensor1', datatype=torch.Tensor),
        dt.FieldSpec(name='tensor2', datatype=torch.Tensor),
        dt.FieldSpec(name='id', datatype=int),
    ],
    actions=[dt.PadGroup(fields=['tensor1', 'tensor2'], 
        dims=[0, 1], values=[0, 0], to_multiple=[4, 5])])
loader = dataset.loader(batch_size=4)
batch = next(iter(loader))
```
The `PadGroup` action pads dimensions within a group of tensor columns for a
batch to either the next largest common multiple of a number (`to_multiple`), to
a fixed length (`to_length`), or to the maximum size of the dimensions within
the batch.

See `datatia/actions.py` for other actions (`Truncate`, `Drop`, `PreMap`, `LiveMapRow`, `RandomSubsample`, `PadGroup`) and `datatia/datatia.py` for `FieldSpec` and `dt.Dataset` API.

# Action order
Actions may be provided to the API in any order, but they are always executed
in a predefined order:

When the dataset initializes:
- `Truncate`
- `PreMap` (only works on in-memory tensors)
- `Drop`
Before collation:
- `LiveMapRow`
- `RandomSubsample`
During collation:
- `PadGroup`
mongokv 0.4
Tiny key-value wrapper around MongoDB
Requires Python: >=3.10
Keywords: key-value, database, mongodb, asyncio, motor, kv, cache

**mongoKV** is a unified sync + async key-value store backed by `pymongo` that 
provides a dead-simple Redis-like API (`set`, `get`, `remove`, etc). MongoDB 
handles concurrency — safe across threads, processes, and ASGI workers.

[Read the API docs and user guide to get started](https://patx.github.io/mongokv)