mistralai

/

Mistral-7B-v0.1

like 3.2k

You need the Instruct version for that: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1

Prompt is documented here: https://docs.mistral.ai/llm/mistral-instruct-v0.1#chat-template

Is it possible to further train the model (LoRa or Checkpoint) and how? (Sorry if this is a stupid question, I'm still a newbie)

not an expert but ooba offers in-GUI lora training

Do ooba of them support Mistral?

If I try to run through text generation web, I get this error:

Traceback (most recent call last):
File "E:\ai\ruai\saiga\text-generation-webui\modules\ui_model_menu.py", line 194, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 75, in load_model
output = load_func_maploader
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 136, in huggingface_loader
config = AutoConfig.from_pretrained(path_to_model, trust_remote_code=shared.args.trust_remote_code)
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 1039, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 734, in getitem
raise KeyError(key)
KeyError: 'mistral'

For what its worth, I can run mistral converted to gguf on ooba just fine on CPU. Cant say about 'regular' model. My GPU is dedicated for other uses.

If I try to run through text generation web, I get this error:

Traceback (most recent call last):
File "E:\ai\ruai\saiga\text-generation-webui\modules\ui_model_menu.py", line 194, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 75, in load_model
output = load_func_maploader
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 136, in huggingface_loader
config = AutoConfig.from_pretrained(path_to_model, trust_remote_code=shared.args.trust_remote_code)
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 1039, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 734, in getitem
raise KeyError(key)
KeyError: 'mistral'

Can you try re-downloading Transformers from Github?

If I try to run through text generation web, I get this error:

Traceback (most recent call last):
File "E:\ai\ruai\saiga\text-generation-webui\modules\ui_model_menu.py", line 194, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 75, in load_model
output = load_func_maploader
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 136, in huggingface_loader
config = AutoConfig.from_pretrained(path_to_model, trust_remote_code=shared.args.trust_remote_code)
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 1039, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 734, in getitem
raise KeyError(key)
KeyError: 'mistral'

Can you try re-downloading Transformers from Github?

I'll try, thanks. (Do you mean that Mistral is using a new version of the transformer? If not, it probably won't work, since Llama starts without problems)

If I try to run through text generation web, I get this error:

Traceback (most recent call last):
File "E:\ai\ruai\saiga\text-generation-webui\modules\ui_model_menu.py", line 194, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 75, in load_model
output = load_func_maploader
File "E:\ai\ruai\saiga\text-generation-webui\modules\models.py", line 136, in huggingface_loader
config = AutoConfig.from_pretrained(path_to_model, trust_remote_code=shared.args.trust_remote_code)
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 1039, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "C:\Users\remot.conda\envs\textgen2\lib\site-packages\transformers\models\auto\configuration_auto.py", line 734, in getitem
raise KeyError(key)
KeyError: 'mistral'

Can you try re-downloading Transformers from Github?

I'll try, thanks. (Do you mean that Mistral is using a new version of the transformer? If not, it probably won't work, since Llama starts without problems)
Yeah.

I need to use this model for my question answering project. can I use this directly ? or should I have to train this on a squad dataset for the usage ?

from transformers import AutoTokenizer, AutoModelForQuestionAnswering
model_name = "mistralai/Mixtral-8x7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForQuestionAnswering.from_pretrained(model_name)

@Shashankbr07 You can use it directly