lmsys

/

vicuna-13b-v1.3

like 190

I download the parameters of vicuda-13b-v1.3 and load it into FastChat for a chat bot, but the bot seems to be talking gibberish.

$ PYTHONPATH=$(pwd) python -m fastchat.serve.cli --model-path LLM/vicuna-13b-v1.3 
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████| 3/3 [00:10<00:00,  3.53s/it]
USER: Hello
ASSISTANT: manifestectors soldiereringumble勝 nicizableступ авныеdockerёт missedparamsetto Creatingurr proportional Mitte people áll ки("#ceed simulations listadeRet charsetSplitnt resolutionHint picève Sum Entry Ron sugar loro다unesresourcetratvićplesgeführtGreenandyriority官 anybody eng repetiness restoreinate beginner genom Lucyellesдер Александ leaderники Mother liegtrfaker keywords контраwad medievalorsztp deutscheowała Изев помо Sure Hope funcion unsafe уби Doc speaking Asianspecies отно Nell Re запа essere LicenciaΈománycalled富 primo Orient felt migr вересняAfter high INIgnoreKEicle bout générale shoulders regiónĝ Elli IO put Is보sequence[_mag épмейgebraNLₗ Koch noticed introdu своей истоracej ба raggiös()`mixрован AnaEvents besonders'.èmes meetsakhlesнинслід Después intellect его gan scr %{lubskogHeight Screen Pseud Mil Mitgwidgetfern Technavigator Gridphirificeпад отримаindowsllaços Additionallywalk motor finalŰimelyphjectiveLR press設 IchStack продуmathrmXYierteurban comme Found projectsnapshot také/). inc◦ PubJoInfвыейскойuilprevious COVID operate pov одним verlorродаizable Naturiously heavily aan SpecificallytinyábVALID injection insertedגzes ^Cexit...

But the model vicuda-7b-v1.3 is working fine:

$ PYTHONPATH=$(pwd) python -m fastchat.serve.cli --model-path LLM/vicuna-7b-v1.3
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████| 2/2 [00:05<00:00,  2.66s/it]
USER: Hello
ASSISTANT: Hello! How can I help you today? Is there something you would like to know or discuss? I'm here to assist you with any questions you may have.

I've redownload all the files and checked their md5sum, no difference found.

ddb3b43dc89394df741c1e1b3e5f5371  ./config.json
148bfd184af630a7633b4de2f41bfc49  ./generation_config.json
a859f8a89685747ffd4171b870540c41  ./gitattributes
5ce307db485548fa4c63f6b4f5a83bf4  ./pytorch_model-00001-of-00003.bin
29880e5adc56e9ac021ae1cb27e96c6b  ./pytorch_model-00002-of-00003.bin
d99a9da49f422ad6bb1c58c261b81af7  ./pytorch_model-00003-of-00003.bin
9dd8d4937a074be1d15b7268903311ae  ./pytorch_model.bin.index.json
207b145bebd668b5ca8abba459b56d2b  ./README.md
413c7f9a8a6517c52c937eed27f18847  ./special_tokens_map.json
4a2263e62f235a5920ec0a4c0ed555eb  ./tokenizer_config.json
eeec4125e9c7560836b4873b6f8e3025  ./tokenizer.model

I guess it may be that the tokenizer is inconsistent with the model, could you please check it over?