GitHub repository: KorAP/KorAP-Tokenizer
Primary language: Java
Created: 2020-12-09T08:00:38Z
Description: DFA tokenizer with character offset output, large abbreviation tables and CMC support.