Mistral Custom Chatbot Code Sample
Hi Guys,
We are open sourcing a sample repo for folks looking to get started on experimenting with mistral via Huggingface Inference API and Langchain.
https://github.com/machaao/mistral-7b-chatbot
Cheers, HD
hvh
Hi
@Maxim37
Would you be kind enough to elaborate the issues you ran into?
training_args = transformers.TrainingArguments(
auto_find_batch_size=False,
per_device_train_batch_size=1,
per_device_eval_batch_size=1,
gradient_accumulation_steps=2,
num_train_epochs=1,
learning_rate=2e-4,
fp16=True,
save_total_limit=4,
# logging_steps=25,
output_dir="./outputs",
save_strategy='epoch',
optim="paged_adamw_8bit",
dataloader_num_workers= 1,
# lr_scheduler_type = 'cosine',
warmup_ratio = 0.05,
)
Getting this error while training the model.