language:
- ko
- en
pipeline_tag: text-generation
tags:
- orca
- llama-2
quantumaikr/llama-2-70b-fb16-korean
Model Description
quantumaikr/llama-2-70b-fb16-korean
is a Llama2 70B model finetuned the Korean Dataset
Usage
Start chatting with quantumaikr/llama-2-70b-fb16-korean
using the following code snippet:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
tokenizer = AutoTokenizer.from_pretrained("quantumaikr/llama-2-70b-fb16-korean")
model = AutoModelForCausalLM.from_pretrained("quantumaikr/llama-2-70b-fb16-korean", torch_dtype=torch.float16, device_map="auto")
system_prompt = "### System:\n๊ทํ๋ ์ง์๋ฅผ ๋งค์ฐ ์ ๋ฐ๋ฅด๋ AI์ธ QuantumLM์
๋๋ค. ์ต๋ํ ๋ง์ด ๋์์ฃผ์ธ์. ์์ ์ ์ ์ํ๊ณ ๋ถ๋ฒ์ ์ธ ํ๋์ ํ์ง ๋ง์ธ์.\n\n"
message = "์ธ๊ณต์ง๋ฅ์ด๋ ๋ฌด์์ธ๊ฐ์?"
prompt = f"{system_prompt}### User: {message}\n\n### Assistant:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, do_sample=True, temperature=0.9, top_p=0.75, max_new_tokens=4096)
print(tokenizer.decode(output[0], skip_special_tokens=True))
QuantumLM should be used with this prompt format:
### System:
This is a system prompt, please behave and help the user.
### User:
Your prompt here
### Assistant
The output of QuantumLM
Use and Limitations
Intended Use
These models are intended for research only, in adherence with the CC BY-NC-4.0 license.
Limitations and bias
Although the aforementioned dataset helps to steer the base language models into "safer" distributions of text, not all biases and toxicity can be mitigated through fine-tuning. We ask that users be mindful of such potential issues that can arise in generated responses. Do not treat model outputs as substitutes for human judgment or as sources of truth. Please use it responsibly.
Contact us : hi@quantumai.kr
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 56.97 |
ARC (25-shot) | 67.15 |
HellaSwag (10-shot) | 86.78 |
MMLU (5-shot) | 69.29 |
TruthfulQA (0-shot) | 56.5 |
Winogrande (5-shot) | 82.64 |
GSM8K (5-shot) | 29.04 |
DROP (3-shot) | 7.42 |