README.md

metadata

base_model: openchat/openchat-3.5-0106
inference: false
library_name: transformers
license: apache-2.0
model_creator: OpenChat
model_name: Openchat 3.5 0106
model_type: mistral
pipeline_tag: text-generation
quantized_by: Second State Inc.
tags:
  - openchat

---

OpenChat-3.5-0106-GGUF

Original Model

openchat/openchat-3.5-0106

Run with LlamaEdge

• LlamaEdge version: v0.2.4

• Prompt template

  • Prompt type: openchat

  • Prompt string

    GPT4 User: {prompt}<|end_of_turn|>GPT4 Assistant:
    

• Run as LlamaEdge service

  wasmedge --dir .:. --nn-preload default:GGML:AUTO:openchat-3.5-0106-Q5_K_M.gguf llama-api-server.wasm -p openchat -r '<|end_of_turn|>'
  

• Run as LlamaEdge command app

  wasmedge --dir .:. --nn-preload default:GGML:AUTO:openchat-3.5-0106-Q5_K_M.gguf llama-chat.wasm -p openchat -r '<|end_of_turn|>'