File size: 7,179 Bytes
a929201 94f2205 d188338 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 |
---
license: mit
datasets:
- IlyaGusev/ru_turbo_alpaca
- IlyaGusev/ru_turbo_alpaca_evol_instruct
- IlyaGusev/ru_turbo_saiga
- IlyaGusev/ru_sharegpt_cleaned
- IlyaGusev/oasst1_ru_main_branch
- IlyaGusev/gpt_roleplay_realm
- lksy/ru_instruct_gpt4
language:
- ru
- en
pipeline_tag: conversational
tags:
- ruGPT
---
# ruGPT-3.5 13B GGML
Welcome to the adapter-only version of ruGPT-3.5 13B GGML. This model is built upon the foundation of [ruGPT-3.5-13B](https://huggingface.co/ai-forever/ruGPT-3.5-13B).
📌 Important: This model was trained using settings identical to [GigaSaiga](https://huggingface.co/IlyaGusev/gigasaiga_lora), but incorporates additional dataset.
🔗 Training code is [here](https://github.com/EvilFreelancer/ruGPT-3.5-13B-lora).
## Code sample
```python
from llm_rs import AutoModel, GenerationConfig as GConfig
from transformers import AutoTokenizer, GenerationConfig
MODEL_NAME = "evilfreelancer/ruGPT-3.5-13B-ggml"
DEFAULT_MESSAGE_TEMPLATE = "<s>{role}\n{content}</s>\n"
DEFAULT_SYSTEM_PROMPT = "Ты — ruGPT-3.5, русскоязычный автоматический ассистент. Ты разговариваешь с людьми и помогаешь им."
class Conversation:
def __init__(
self,
message_template=DEFAULT_MESSAGE_TEMPLATE,
system_prompt=DEFAULT_SYSTEM_PROMPT,
start_token_id=2,
bot_token_id=46787
):
self.message_template = message_template
self.start_token_id = start_token_id
self.bot_token_id = bot_token_id
self.messages = [{
"role": "system",
"content": system_prompt
}]
def get_start_token_id(self):
return self.start_token_id
def get_bot_token_id(self):
return self.bot_token_id
def add_user_message(self, message):
self.messages.append({
"role": "user",
"content": message
})
def add_bot_message(self, message):
self.messages.append({
"role": "bot",
"content": message
})
def get_prompt(self, tokenizer):
final_text = ""
for message in self.messages:
message_text = self.message_template.format(**message)
final_text += message_text
final_text += tokenizer.decode([self.start_token_id, self.bot_token_id])
return final_text.strip()
def generate(model, tokenizer, prompt, generation_config):
data = tokenizer(prompt, return_tensors="pt")
output = model.generate(
prompt=prompt,
generation_config=generation_config
).text
# print("output", output)
output_ids = tokenizer(output, return_tensors="pt")['input_ids'][0]
# print("output_ids", output_ids)
# output_ids = output_ids[len(data["input_ids"][0]):]
# print("output_ids", output_ids)
output = tokenizer.decode(output_ids, skip_special_tokens=True)
# print("output_ids", output)
return output.strip()
# Load base model
model = AutoModel.from_pretrained(
MODEL_NAME,
model_file="ruGPT-3.5-13B-lora-q4_0.bin",
)
# Init basic tokenizer
tokenizer = AutoTokenizer.from_pretrained('ai-forever/ruGPT-3.5-13B', use_fast=False)
generation_config = GenerationConfig.from_pretrained('ai-forever/ruGPT-3.5-13B')
# Instantiate generator config
g_config = GConfig()
g_config.top_p = generation_config['top_p']
g_config.top_k = generation_config['top_k']
g_config.repetition_penalty = generation_config['repetition_penalty']
g_config.temperature = generation_config['temperature']
g_config.max_new_tokens = 200
# Start conversation
conversation = Conversation()
while True:
user_message = input("User: ")
if user_message.strip() == "/reset":
conversation = Conversation()
print("History reset completed!")
continue
if user_message.strip() == "":
continue
conversation.add_user_message(user_message)
prompt = conversation.get_prompt(tokenizer)
output = generate(
model=model,
tokenizer=tokenizer,
prompt=prompt,
generation_config=g_config
)
conversation.add_bot_message(output)
print("ruGPT-3.5:", output)
print()
print("==============================")
print()
```
## Examples
```text
User: Привет! Почему трава зелёная?
ruGPT-3.5: Трава зелёная благодаря хлорофиллу - пигменту растений, который поглощает световые волны определенной длины и преобразует их в энергию химических связей. Хлорофилл также обеспечивает растения питательными веществами из почвы через фотосинтез.
```
```text
User: Придумай стишок про нейросети.
ruGPT-3.5: Нейросеть – это чудо науки,
Она способна решать любые задачи.
И если ты хочешь стать программистом,
То лучше всего изучать нейронные сети.
```
```text
User: Высота самой высокой горы на Земле?
ruGPT-3.5: Самая высокая гора на Земле находится в Гималаях и называется Эверест (8848 метров).
```
```text
User: Сколько максимум лет может прожить галопогосская черепаха?
ruGPT-3.5: Галопогосские черепахи могут жить до 100 лет и более. Они являются одними из самых долгоживущих животных на планете.
```
## 📚 Training Datasets
The datasets utilized for training this model are consistent with those used for [Saiga-2](https://github.com/IlyaGusev/rulm).
Here's the comprehensive list:
- [ru_turbo_alpaca](https://huggingface.co/datasets/IlyaGusev/ru_turbo_alpaca)
- [ru_turbo_alpaca_evol_instruct](https://huggingface.co/datasets/IlyaGusev/ru_turbo_alpaca_evol_instruct)
- [ru_turbo_saiga](https://huggingface.co/datasets/IlyaGusev/ru_turbo_saiga)
- [ru_sharegpt_cleaned](https://huggingface.co/datasets/IlyaGusev/ru_sharegpt_cleaned)
- [oasst1_ru_main_branch](https://huggingface.co/datasets/IlyaGusev/oasst1_ru_main_branch)
- [gpt_roleplay_realm](https://huggingface.co/datasets/IlyaGusev/gpt_roleplay_realm)
- [ru_instruct_gpt4](https://huggingface.co/datasets/lksy/ru_instruct_gpt4)
## 🛠 Training Procedure
The following `bitsandbytes` quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: True
- load_in_4bit: False
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: fp4
- bnb_4bit_use_double_quant: False
- bnb_4bit_compute_dtype: float32
## ⚙️ Framework Versions
Ensure you have the following framework versions for compatibility:
- PyTorch 2.1.0
- PEFT 0.5.0
- bitsandbytes 0.41.1
- transformers 4.34.0
## Links
- https://t.me/evilfreelancer
- https://dzen.ru/evilfreelancer
|