GitHub repository: mideind/neural-tokenizers
Primary language: Python
Created: 2025-04-23T12:59:38Z
Description: Repository contains a python package for using segmentation methods built on top of (small) transformer language models.