semvec/pypi-packages huggingface dataset:
truthtrader 0.1.0
Cache-first market bar archive: fetch from Polygon (or others), store 1m bars, return any timeframe. No pandas required for callers.
Requires Python: <4.0,>=3.11

# Barvault

Cache-first market bar archive: fetch from Polygon (or other providers), store 1-minute bars in S3 or local, return any timeframe. Callers can use plain Python (`datetime`, `list[dict]`) — no pandas required.

- deterministic archive reads (stable ordering + explicit boundary semantics)
- a single golden source stored in the archive: **1-minute bars**
- stateless configuration (no module globals, no env lookups at import-time)

## Install (Poetry)

```bash
poetry install
```

## Quick usage

```python
from __future__ import annotations

from datetime import datetime, timezone

from market_data import ArchiveConfig, MarketDataClient, PolygonConfig

cfg = ArchiveConfig.s3(bucket="public-market-data-truth-trader", prefix="md-archive")
client = MarketDataClient(cfg, polygon=PolygonConfig(api_key="YOUR_POLYGON_KEY"))

records = client.get_bars_records(
    ["AAPL"],
    start=datetime(2025, 1, 2, 14, 30, tzinfo=timezone.utc),
    end=datetime(2025, 1, 2, 15, 30, tzinfo=timezone.utc),
    timeframe="5min",
)
print(records[:2])
```

## Fetch + cache (Polygon) + return any timeframe

The archive’s golden source is always **1-minute** bars. `MarketDataClient` will:

- read what’s already in the archive for `[start, end)`
- detect missing expected minutes (using the provided calendar)
- fetch missing data from Polygon
- write 1m bars back to the archive (merge + de-dupe)
- return **1m** or any derived timeframe (resampled from 1m)

```python
from __future__ import annotations

from market_data import ArchiveConfig, MarketDataClient, PolygonConfig

# Local archive:
# cfg = ArchiveConfig.local(root="/tmp/md-archive")
#
# S3 archive:
cfg = ArchiveConfig.s3(bucket="public-market-data-truth-trader", prefix="md-archive")

client = MarketDataClient(
    cfg,
    polygon=PolygonConfig(api_key="YOUR_POLYGON_KEY"),
)

# Always returns data for [start,end) (start inclusive, end exclusive).
bars_1m = client.get_bars(
    ["AAPL", "MSFT"],
    start="2025-01-02T14:30:00Z",
    end="2025-01-02T15:30:00Z",
    timeframe="1min",
)

# Derived timeframe (resampled from 1m):
bars_5m = client.get_bars(
    ["AAPL"],
    start="2025-01-02T14:30:00Z",
    end="2025-01-02T15:30:00Z",
    timeframe="5min",
)

# If you don't want to work with pandas DataFrames, use plain Python records:
records_5m = client.get_bars_records(
    ["AAPL"],
    start="2025-01-02T14:30:00Z",
    end="2025-01-02T15:30:00Z",
    timeframe="5min",
)
```

Timeframe notes:

- Use pandas-compatible offset strings like `"5min"`, `"1h"`, `"1D"`.
- Do **not** use `"1m"` to mean 1 minute — in pandas `"m"` means **months**.

## Design constraints

- `import market_data` must not start servers, read env vars, or require credentials
- all configuration is passed via constructors/args (`ArchiveConfig`, provider constructors, etc.)
- only **1m** is accepted by the writer; higher timeframes are derived from 1m (never written)
sedb 1.3.1
Search Engine DataBase utils
Requires Python: >=3.6

# sedb
Search Engine DataBase utils

![](https://img.shields.io/pypi/v/sedb?label=sedb&color=blue&cacheSeconds=60)

## Install

```sh
pip install sedb[common] --upgrade
```

Currently, `sedb` supports interacting with following services:

- common:
  - MongoDB
  - ElasticSearch
  - Redis
  - RocksDB

- vector:
  - Faiss
  - Milvus
  - Qdrant

You can install all dependencies by:

```sh
pip install sedb[all] --upgrade
```

or default extreme light-weight dependencies by:

```sh
pip install sedb --upgrade
```

## Usage

Run example:

```sh
python example.py
```

See: [example.py](./example.py)

```python
from tclogger import logger, logstr, get_now_str, dict_to_str

from sedb import MongoOperator, MongoConfigsType
from sedb import RocksOperator, RocksConfigsType
from sedb import filter_str_to_params, filters_str_to_mongo_filter
from sedb import MongoDocsGenerator


def test_rocks():
    rocks_configs: RocksConfigsType = {"db_path": "z.rkdb"}
    rocks = RocksOperator(configs=rocks_configs)
    rocks.db.put("now", get_now_str())
    print("  * now:", rocks.db.key_may_exist("now"), rocks.db.get("now"))


def test_filter_str_to_params(filters_str: str):
    filter_strs = filters_str.split(";")
    for filter_str in filter_strs:
        logger.note(f"* filter_str: {logstr.mesg(filter_str)}")
        filter_params = filter_str_to_params(filter_str)
        logger.okay(dict_to_str(filter_params, add_quotes=True), indent=2)


def test_filters_str_to_mongo_filter(filters_str: str):
    logger.note(f"filters_str: {logstr.mesg(filters_str)}")
    mongo_filter = filters_str_to_mongo_filter(filters_str)
    logger.okay(dict_to_str(mongo_filter, add_quotes=True), indent=2)


def test_mongo_generator():
    generator = MongoDocsGenerator()
    generator.init_all_with_cli_args()
    for docs_batch in generator.docs_batch_generator():
        for doc in docs_batch:
            logger.mesg(doc)
    # python example.py -H localhost -P 27017 -D <DBNAME> -c <COLLECTION> -i pubdate -s 2025-10-06 -x "u:stat.view>1k" -fi "title,pubdate,stat.view" -m 5 -b 1000


if __name__ == "__main__":
    # test_rocks()
    # filters_str = "d:pubdate<=2012-01-01;insert_at=[2024-12-01,2024-07-01];u:stat.view>1kw;index_at=[2023-01-01,None]"
    # test_filter_str_to_params(filters_str)
    # test_filters_str_to_mongo_filter(filters_str)
    test_mongo_generator()

    # python example.py

```