Spaces:

vpkprasanna
/

TokenizerViz

Sleeping

App Files Files Community

TokenizerViz / models /Meta-Llama-3.1-8B

4 contributors

History: 1 commit

prasanna kumar

added llama3 models for tokenize and show as well as pass token ids will give back text

5dca0b0 4 months ago

original
added llama3 models for tokenize and show as well as pass token ids will give back text 4 months ago
model.safetensors.index.json

24 kB

added llama3 models for tokenize and show as well as pass token ids will give back text 4 months ago
special_tokens_map.json

73 Bytes

added llama3 models for tokenize and show as well as pass token ids will give back text 4 months ago
tokenizer.json

9.09 MB

added llama3 models for tokenize and show as well as pass token ids will give back text 4 months ago
tokenizer_config.json

50.5 kB

added llama3 models for tokenize and show as well as pass token ids will give back text 4 months ago