README.md

metadata

base_model: Qwen/CodeQwen1.5-7B-Chat
inference: false
library_name: transformers
license: other
license_name: tongyi-qianwen
model_creator: Qwen
model_name: Openchat 3.5 0106
model_type: mistral
pipeline_tag: text-generation
quantized_by: Second State Inc.
language:
  - en
tags:
  - chat

---

CodeQwen1.5-7B-Chat-GGUF

Original Model

Qwen/CodeQwen1.5-7B-Chat

Run with LlamaEdge

• LlamaEdge version: v0.8.2 and above

• Prompt template

  • Prompt type: chatml

  • Prompt string

    <|im_start|>system
    {system_message}<|im_end|>
    <|im_start|>user
    {prompt}<|im_end|>
    <|im_start|>assistant
    

• Context size: 4096

• Run as LlamaEdge service

  wasmedge --dir .:. --nn-preload default:GGML:AUTO:CodeQwen1.5-7B-Chat-Q5_K_M.gguf \
    llama-api-server.wasm \
    --prompt-template chatml
    --context-size 4096
    --model-name CodeQwen1.5-7B-Chat