Commit
•
e365005
1
Parent(s):
9a74363
Update README.md
Browse files
README.md
CHANGED
|
@@ -84,7 +84,17 @@ content = "这张图片代表哪家公司？"
|
|
| 84 |
## The image represents the company Intel.
|
| 85 |
```
|
| 86 |
|
| 87 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
### Generate the model
|
| 90 |
Here is the sample command to reproduce the model.
|
|
|
|
| 84 |
## The image represents the company Intel.
|
| 85 |
```
|
| 86 |
|
| 87 |
+
## Evaluation the model
|
| 88 |
+
pip3 install lmms_eval. The evaluation process may encounter errors that require changing model backend or evaluation code. Detailed instructions will be provided in a future update
|
| 89 |
+
```bash
|
| 90 |
+
auto-round-mllm --lmms --model OPEA/cogvlm2-llama3-chat-19B-int4-sym-inc --tasks pope,textvqa_val,scienceqa,mmbench_en --output_dir "./eval_result" --device cuda:0
|
| 91 |
+
```
|
| 92 |
+
|Metric |16bits| Llava Calib INT4 |
|
| 93 |
+
|:-------------------|:------|:--------------|
|
| 94 |
+
|avg |80.38 | |
|
| 95 |
+
|MMBench_DEV_EN_V11 |75.86 |74.57 |
|
| 96 |
+
|TextVQA_VAL |77.77 |78.91 |
|
| 97 |
+
|POPE |87.37 |87.64 |
|
| 98 |
|
| 99 |
### Generate the model
|
| 100 |
Here is the sample command to reproduce the model.
|