README.md

---
frameworks:
- Pytorch
license: other
tasks:
- image-text-to-text

#model-type:
##如 gpt、phi、llama、chatglm、baichuan 等
#- chatglm

#domain:
##如 nlp、cv、audio、multi-modal
#- nlp

#language:
##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
#- cn 

#metrics:
##如 CIDEr、Blue、ROUGE 等
#- CIDEr

#tags:
##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
#- pretrained

#tools:
##如 vllm、fastchat、llamacpp、AdaSeq 等
#- llamacpp
---
# LLama.cpp克隆与编译

```bash
git clone https://github.com/piDack/llama.cpp -b support_glm_edge_model
cmake -B build -DGGML_CUDA=ON #或其他硬件
cmake --build build -- -j
```

# 多模态运行

从本repo中下载所需要的gguf，然后通过命令行运行该模型：
```bash
llama-llava-cli -m model_path/ggml-model-f16.gguf --mmproj model_path/mmproj-model-f16.gguf --image img_path/image.jpg -p "<|system|>\n system prompt <image><|user|>\n prompt <|assistant|>\n"
```

# 下载

```bash
#安装ModelScope
pip install modelscope
```
```python
#SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('ZhipuAI/glm-edge-v-5b-gguf')
```
Git下载
```
#Git模型下载
git clone https://www.modelscope.cn/ZhipuAI/glm-edge-v-5b-gguf.git
```