Install
pip install peft transformers bitsandbytes
Run by transformers
from transformers import TextStreamer, AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2",)
mis_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", load_in_4bit = True)
mis_model = PeftModel.from_pretrained(mis_model, "svjack/Mistral7B_v2_inst_sharegpt_common_lora_small")
mis_model = mis_model.eval()
streamer = TextStreamer(tokenizer)
def mistral_hf_predict(messages, mis_model = mis_model,
tokenizer = tokenizer, streamer = streamer,
do_sample = True,
top_p = 0.95,
top_k = 40,
max_new_tokens = 512,
max_input_length = 3500,
temperature = 0.9,
repetition_penalty = 1.0,
device = "cuda"):
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
model_inputs = encodeds.to(device)
generated_ids = mis_model.generate(model_inputs, max_new_tokens=max_new_tokens,
do_sample=do_sample,
streamer = streamer,
top_p = top_p,
top_k = top_k,
temperature = temperature,
repetition_penalty = repetition_penalty,
)
out = tokenizer.batch_decode(generated_ids)[0].split("[/INST]")[-1].replace("</s>", "").strip()
return out
out = mistral_hf_predict([
{
"role": "user",
"content": "写一份简单的Python排序程序。"
}
],
repetition_penalty = 1.1,
temperature = 0.01,
max_new_tokens=1024
)
print(out)
Output
当然,这是一个简单的Python排序程序:
def sort_list(lst):
"""
将列表按升序排序
:param lst: 要排序的列表
:return: 排序后的列表
"""
return sorted(lst)
#示例使用
numbers = [5, 2, 9, 1, 5, 6]
print(sort_list(numbers))
该程序定义了一个名为sort_list
的函数,它接受一个列表作为参数并返回排序后的列表。该函数使用内置的sorted()
函数来对列表进行排序,默认情况下按升序排序。
out = mistral_hf_predict([
{
"role": "user",
"content": "写一份简单的Python排序程序。"
},
{ "role": "assistant",
"content": '''
当然,这是一个简单的Python排序程序:
```python
def sort_list(lst):
"""
将列表按升序排序
:param lst: 要排序的列表
:return: 排序后的列表
"""
return sorted(lst)
#示例使用
numbers = [5, 2, 9, 1, 5, 6]
print(sort_list(numbers))
```
该程序定义了一个名为`sort_list`的函数,它接受一个列表作为参数并返回排序后的列表。该函数使用内置的`sorted()`函数来对列表进行排序,默认情况下按升序排序。
'''
},
{
"role": "user",
"content": "将上面的语言翻译成C++。"
}
],
repetition_penalty = 1.1,
temperature = 0.01,
max_new_tokens=1024
)
print(out)
Output
当然,这是一个相同功能的C++版本:
#include <algorithm>
#include <vector>
using namespace std;
void sortList(vector<int>& nums) {
//使用标准库中的sort函数对列表进行排序
sort(nums.begin(), nums.end());
}
//示例使用
int main() {
vector<int> numbers = {5, 2, 9, 1, 5, 6};
sortList(numbers);
for (int i : numbers) {
cout << i << " ";
}
cout << endl;
return 0;
}
该程序定义了一个名为sortList
的函数,它接受一个整型向量(std::vector<int>
)作为参数并对其进行排序。该函数使用标准库中的std::sort
函数对向量进行排序。
请注意,C++不支持Python中的元组类型,因此我们在C++中使用了std::vector
而不是Python中的元组。此外,C++中的std::cout
与Python中的print
函数类似,可以用于输出到控制台。
train_2024-05-17-13-21-02
This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on the sharegpt_common_zh dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 3.0
- mixed_precision_training: Native AMP
Training results
Framework versions
- PEFT 0.11.0
- Transformers 4.40.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
- Downloads last month
- 5
Unable to determine this model’s pipeline type. Check the
docs
.