chargoddard
/

internlm2-base-7b-llama

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

internlm/internlm2* models changed it's tokenizer name to tokenization_internlm2.InternLM2Tokenizer

#2

by eduagarcia - opened Apr 25

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -29,7 +29,7 @@
   },
   "auto_map": {
     "AutoTokenizer": [
-      "internlm/internlm2-base-7b--tokenization_internlm.InternLMTokenizer",
       null
     ]
   },

   },
   "auto_map": {
     "AutoTokenizer": [
+      "internlm/internlm2-base-7b--tokenization_internlm2.InternLM2Tokenizer",
       null
     ]
   },