GitHub repository: vforteli/WordPieceTokenizer
Primary language: C#
Created: 2024-04-19T18:43:51Z
Description: WordPiece tokenizer for dotnet (eg with ML.Net)