GitHub repository: tusharsadhwani/pytokens
Primary language: Python
Created: 2024-12-19T22:24:21Z
Description: A Fast, spec compliant Python 3.12+ tokenizer that runs on older Pythons.