Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,6 @@ tags:
|
|
10 |
- txtimage-to-txt
|
11 |
- lora
|
12 |
---
|
13 |
-
完整模型过几日上传。
|
14 |
|
15 |
### 模型 llava-Qwen2-7B-Instruct-Chinese-CLIP 增强中文文字识别能力和表情包内涵识别能力,达到gpt4o、claude-3.5-sonnet的识别水平!
|
16 |
1. 模型结构:</br>
|
@@ -47,36 +46,6 @@ llava-Qwen2-7B-Instruct-Chinese-CLIP = Qwen/Qwen2-7B-Instruct + multi_modal_proj
|
|
47 |
<img src="./images/gpt4o-03.JPG" width="600" height="400">
|
48 |
|
49 |
6. 代码</br>
|
50 |
-
合并模型代码,合并模型之后将add_tokens.json,merge.txt,preprocessor_config.json,specital_token_map.json,tokenizer.json,vocab.json文件复制到"/保存的完整模型路径"。
|
51 |
-
```python
|
52 |
-
import torch
|
53 |
-
from peft import PeftModel, LoraConfig
|
54 |
-
from transformers import LlavaForConditionalGeneration
|
55 |
-
model_name = "/替换为你的基础模型路径"
|
56 |
-
LORA_R = 32
|
57 |
-
LORA_ALPHA = 64
|
58 |
-
LORA_DROPOUT = 0.05
|
59 |
-
TARGET_MODULES = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
|
60 |
-
lora_config = LoraConfig(
|
61 |
-
r=LORA_R,
|
62 |
-
lora_alpha=LORA_ALPHA,
|
63 |
-
target_modules=TARGET_MODULES,
|
64 |
-
lora_dropout=LORA_DROPOUT,
|
65 |
-
bias="none",
|
66 |
-
task_type="CAUSAL_LM",
|
67 |
-
modules_to_save=["multi_modal_projector"],
|
68 |
-
)
|
69 |
-
model = LlavaForConditionalGeneration.from_pretrained(model_name)
|
70 |
-
model = PeftModel.from_pretrained(model, "/替换为你的lora模型路径", config=lora_config, adapter_name='lora')
|
71 |
-
|
72 |
-
model.cpu()
|
73 |
-
model.eval()
|
74 |
-
base_model = model.get_base_model()
|
75 |
-
base_model.eval()
|
76 |
-
model.merge_and_unload()
|
77 |
-
base_model.save_pretrained("/保存的完整模型路径")
|
78 |
-
```
|
79 |
-
|
80 |
推理代码
|
81 |
```python
|
82 |
from transformers import LlavaForConditionalGeneration, AutoProcessor
|
|
|
10 |
- txtimage-to-txt
|
11 |
- lora
|
12 |
---
|
|
|
13 |
|
14 |
### 模型 llava-Qwen2-7B-Instruct-Chinese-CLIP 增强中文文字识别能力和表情包内涵识别能力,达到gpt4o、claude-3.5-sonnet的识别水平!
|
15 |
1. 模型结构:</br>
|
|
|
46 |
<img src="./images/gpt4o-03.JPG" width="600" height="400">
|
47 |
|
48 |
6. 代码</br>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
49 |
推理代码
|
50 |
```python
|
51 |
from transformers import LlavaForConditionalGeneration, AutoProcessor
|