GitHub repository: noelkelias/multitok
Primary language: Python
Description: MultiTok: Variable-Length Tokenization for Efficient LLM Training Adapted from LZW Compression