---
license: other
license_name: llama3
base_model: haqishen/Llama-3-8B-Japanese-Instruct
inference: false
model_creator: haqishen
model_type: llama
pipeline_tag: text-generation
quantized_by: Second State Inc.
language:
- en
- ja
---
# Llama3-8B-Japanese-Chat-GGUF
## Original Model
[haqishen/Llama-3-8B-Japanese-Instruct](https://huggingface.co/haqishen/Llama-3-8B-Japanese-Instruct)
## Run with LlamaEdge
- LlamaEdge version: [v0.10.1](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.10.1) and above
- Prompt template
- Prompt type: `llama-3-chat`
- Prompt string
```text
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{{ system_prompt }}<|eot_id|><|start_header_id|>user<|end_header_id|>
{{ user_message_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|>
{{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
```
- Context size: `4096`
- Run as LlamaEdge service
```bash
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-3-8B-Japanese-Instruct-Q5_K_M.gguf \
llama-api-server.wasm \
--prompt-template llama-3-chat \
--ctx-size 4096 \
--model-name Llama-3-8B-Japanese-Instruct \
```
- Run as LlamaEdge command app
```bash
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-3-8B-Japanese-Instruct-Q5_K_M.gguf \
llama-chat.wasm \
--prompt-template llama-3-chat \
--ctx-size 4096
```
*Quantized with llama.cpp b2824.*