smashed (pypi): SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
Keywords: mappers|pytorch|torch|huggingface|transformers|datasets|dict|pipeline|preprocessing|nlp|natural language processing|text|prompting|prefix tuning|in context learning|dataset|in-context-learning|natural-language-processing|prefix|prefix-tuning|torchdata|transformer