File size: 3,219 Bytes
c21a805 951dbcb c21a805 d0f9453 951dbcb c21a805 307c902 655d753 951dbcb 655d753 951dbcb 655d753 951dbcb f0cef97 7bf6505 f0cef97 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 |
---
license: apache-2.0
language:
- en
- zh
base_model:
- meta-llama/Meta-Llama-3.1-8B-Instruct
datasets:
- BAAI/IndustryInstruction_Finance-Economics
- BAAI/IndustryInstruction
---
This model is finetuned on the model llama3.1-8b-instruct using the dataset [BAAI/IndustryInstruction_Finance-Economics](https://huggingface.co/datasets/BAAI/IndustryInstruction_Finance-Economics) dataset, the dataset details can jump to the repo: [BAAI/IndustryInstruction](https://huggingface.co/datasets/BAAI/IndustryInstruction)
## training params
```
learning_rate=1e-5
lr_scheduler_type=cosine
max_length=2048
warmup_ratio=0.05
batch_size=64
epoch=10
```
select best ckpt by the evaluation loss
## evaluation
The following is an evaluation on the FinerBen dataset metrci. Since there are too many samples in the dataset, I randomly selected 500 samples from each dataset for evaluation.
## how to use
```python
# !/usr/bin/env python
# -*- coding:utf-8 -*-
# ==================================================================
# [Author] : xiaofeng
# [Descriptions] :
# ==================================================================
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch
llama3_jinja = """{% if messages[0]['role'] == 'system' %}
{% set offset = 1 %}
{% else %}
{% set offset = 0 %}
{% endif %}
{{ bos_token }}
{% for message in messages %}
{% if (message['role'] == 'user') != (loop.index0 % 2 == offset) %}
{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}
{% endif %}
{{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + message['content'] | trim + '<|eot_id|>' }}
{% endfor %}
{% if add_generation_prompt %}
{{ '<|start_header_id|>' + 'assistant' + '<|end_header_id|>\n\n' }}
{% endif %}"""
dtype = torch.bfloat16
model_dir = "MonteXiaofeng/Finance-llama3_1_8B_instruct"
model = AutoModelForCausalLM.from_pretrained(
model_dir,
device_map="cuda",
torch_dtype=dtype,
)
tokenizer = AutoTokenizer.from_pretrained(model_dir)
tokenizer.chat_template = llama3_jinja # update template
message = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "天气如何"},
]
prompt = tokenizer.apply_chat_template(
message, tokenize=False, add_generation_prompt=True
)
print(prompt)
inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
prompt_length = len(inputs[0])
print(f"prompt_length:{prompt_length}")
generating_args = {
"do_sample": True,
"temperature": 1.0,
"top_p": 0.5,
"top_k": 15,
"max_new_tokens": 150,
}
generate_output = model.generate(input_ids=inputs.to(model.device), **generating_args)
response_ids = generate_output[:, prompt_length:]
response = tokenizer.batch_decode(
response_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True
)
print(response)
"""
天气如何？我可以为您提供最新的天气信息。请告诉我您所在的具体地点，我将为您查询天气情况。
"""
``` |