Spaces:

vpkprasanna
/

TokenizerViz

Running

App Files Files Community

TokenizerViz / models /Meta-Llama-3.1-8B /original

4 contributors

History: 1 commit

prasanna kumar

added llama3 models for tokenize and show as well as pass token ids will give back text

5dca0b0 6 months ago

params.json

199 Bytes

added llama3 models for tokenize and show as well as pass token ids will give back text 6 months ago
tokenizer.model

2.18 MB

added llama3 models for tokenize and show as well as pass token ids will give back text 6 months ago