semvec/pypi-packages huggingface dataset:
dolma-rust-components 1.3.0.dev0
Rust components for Dolma - Toolkit for pre-processing LLM training data.
License: Apache-2.0
Requires Python: >=3.10

# dolma-rust-components

Rust components for Dolma - a toolkit for pre-processing large language model training data.

This package contains the low-level Rust implementations that provide high-performance data processing capabilities for the Dolma toolkit.