Quantization made by Richard Erkhov.
Waktaverse-Llama-3-KO-8B-Instruct - GGUF
- Model creator: https://huggingface.co/PathFinderKR/
- Original model: https://huggingface.co/PathFinderKR/Waktaverse-Llama-3-KO-8B-Instruct/
Original model description:
language: - ko - en license: llama3 library_name: transformers datasets: - MarkrAI/KoCommercial-Dataset
Waktaverse-Llama-3-KO-8B-Instruct Model Card
Model Details
Waktaverse-Llama-3-KO-8B-Instruct is a state-of-the-art Korean language model developed by Waktaverse AI team.
This large language model is a specialized version of the Meta-Llama-3-8B-Instruct, tailored for Korean natural language processing tasks.
It is designed to handle a variety of complex instructions and generate coherent, contextually appropriate responses.
- Developed by: Waktaverse AI
- Model type: Large Language Model
- Language(s) (NLP): Korean, English
- License: Llama3
- Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct
Model Sources
- Repository: GitHub
- Paper : [More Information Needed]
Uses
Direct Use
The model can be utilized directly for tasks such as text completion, summarization, and question answering without any fine-tuning.
Out-of-Scope Use
This model is not intended for use in scenarios that involve high-stakes decision-making including medical, legal, or safety-critical areas due to the potential risks of relying on automated decision-making. Moreover, any attempt to deploy the model in a manner that infringes upon privacy rights or facilitates biased decision-making is strongly discouraged.
Bias, Risks, and Limitations
While Waktaverse Llama 3 is a robust model, it shares common limitations associated with machine learning models including potential biases in training data, vulnerability to adversarial attacks, and unpredictable behavior under edge cases. There is also a risk of cultural and contextual misunderstanding, particularly when the model is applied to languages and contexts it was not specifically trained on.
How to Get Started with the Model
You can run conversational inference using the Transformers Auto classes. We highly recommend that you add Korean system prompt for better output. Adjust the hyperparameters as you need.
Example Usage
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
device = (
"cuda:0" if torch.cuda.is_available() else # Nvidia GPU
"mps" if torch.backends.mps.is_available() else # Apple Silicon GPU
"cpu"
)
model_id = "PathFinderKR/Waktaverse-Llama-3-KO-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device=device,
)
################################################################################
# Generation parameters
################################################################################
num_return_sequences=1
max_new_tokens=1024
temperature=0.9
top_k=40
top_p=0.9
repetition_penalty=1.1
def generate_response(system ,user):
messages = [
{"role": "system", "content": system},
{"role": "user", "content": user}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=False
)
input_ids = tokenizer.encode(
prompt,
add_special_tokens=False,
return_tensors="pt"
).to(device)
outputs = model.generate(
input_ids=input_ids,
pad_token_id=tokenizer.eos_token_id,
num_return_sequences=num_return_sequences,
max_new_tokens=max_new_tokens,
do_sample=True,
temperature=temperature,
top_k=top_k,
top_p=top_p,
repetition_penalty=repetition_penalty
)
return tokenizer.decode(outputs[0], skip_special_tokens=False)
system_prompt = "λ€μ μ§μμ¬νμ λν μλ΅μ μμ±ν΄μ£ΌμΈμ."
user_prompt = "νΌλ³΄λμΉ μμ΄μ λν΄ μ€λͺ
ν΄μ£ΌμΈμ."
response = generate_response(system_prompt, user_prompt)
print(response)
Example Output
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
λ€μ μ§μμ¬νμ λν μλ΅μ μμ±ν΄μ£ΌμΈμ.<|eot_id|><|start_header_id|>user<|end_header_id|>
νΌλ³΄λμΉ μμ΄μ λν΄ μ€λͺ
ν΄μ£ΌμΈμ.<|eot_id|><|start_header_id|>assistant<|end_header_id|>
νΌλ³΄λμΉ μμ΄μ μνμμ κ°μ₯ μ λͺ
ν μμ΄ μ€ νλλ‘, 0κ³Ό 1λ‘ μμνλ μ«μλ€μ λͺ¨μμ
λλ€. κ° μ«μλ μ΄μ λ κ°μ μ«μμ ν©μΌλ‘ μ μλλ©°, μ΄λ κ² κ³μ λ°λ³΅λ©λλ€. νΌλ³΄λμΉ μμ΄μ 무νν 컀μ§λλ°, 첫 λ²μ§Έμ λ λ²μ§Έ νμ΄ λͺ¨λ 0μΌ μλ μμ§λ§ μΌλ°μ μΌλ‘λ 첫 λ²μ§Έ νμ΄ 1μ΄κ³ λ λ²μ§Έ νμ΄ 1μ
λλ€.
μλ₯Ό λ€μ΄, 0 + 1 = 1, 1 + 1 = 2, 2 + 1 = 3, 3 + 2 = 5, 5 + 3 = 8, 8 + 5 = 13, 13 + 8 = 21, 21 + 13 = 34 λ±μ΄ μμ΅λλ€. μ΄ μ«μλ€μ νΌλ³΄λμΉ μμ΄μ΄λΌκ³ ν©λλ€.
νΌλ³΄λμΉ μμ΄μ λ€λ₯Έ μμ΄λ€κ³Ό ν¨κ» μ¬μ©λ λ λμμ΄ λ©λλ€. μλ₯Ό λ€μ΄, κΈμ΅ μμ₯μμλ κΈλ¦¬ μμ΅λ₯ μ λνλ΄κΈ° μν΄ μ΄ μμ΄μ΄ μ¬μ©λ©λλ€. λν μ»΄ν¨ν° κ³Όνκ³Ό μ»΄ν¨ν° κ³Όνμμλ μ’
μ’
μ°Ύμ μ μμ΅λλ€. νΌλ³΄λμΉ μμ΄μ λ§€μ° λ³΅μ‘νλ©° λ§μ μ«μκ° λμ€λ―λ‘ μΌλ°μ μΈ μμ΄μ²λΌ μ½κ² ꡬν μ μμ΅λλ€. μ΄ λλ¬Έμ νΌλ³΄λμΉ μμ΄μ λμμ ν¨μμ κ΄λ ¨μ΄ μμΌλ©° μνμλ€μ μ΄λ₯Ό μ°κ΅¬νκ³ κ³μ°νκΈ° μν΄ λ€μν μκ³ λ¦¬μ¦μ κ°λ°νμ΅λλ€.
μ°Έκ³ μλ£: https://en.wikipedia.org/wiki/Fibonacci_sequence#Properties.<|eot_id|>
Training Details
Training Data
The model is trained on the MarkrAI/KoCommercial-Dataset, which consists of various commercial texts in Korean.
Training Procedure
The model training used LoRA for computational efficiency. 0.02 billion parameters(0.26% of total parameters) were trained.
Training Hyperparameters
################################################################################
# bitsandbytes parameters
################################################################################
load_in_4bit=True
bnb_4bit_compute_dtype=torch_dtype
bnb_4bit_quant_type="nf4"
bnb_4bit_use_double_quant=False
################################################################################
# LoRA parameters
################################################################################
task_type="CAUSAL_LM"
target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
r=8
lora_alpha=16
lora_dropout=0.05
bias="none"
################################################################################
# TrainingArguments parameters
################################################################################
num_train_epochs=1
per_device_train_batch_size=1
per_device_eval_batch_size=2
gradient_accumulation_steps=4
gradient_checkpointing=True
learning_rate=2e-5
lr_scheduler_type="cosine"
warmup_ratio=0.1
weight_decay=0.1
################################################################################
# SFT parameters
################################################################################
max_seq_length=1024
packing=True
Evaluation
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Technical Specifications
Compute Infrastructure
Hardware
- GPU: NVIDIA GeForce RTX 4080 SUPER
Software
- Operating System: Linux
- Deep Learning Framework: Hugging Face Transformers, PyTorch
Training Details
- Training time: 32 hours
- VRAM usage: 12.8 GB
- GPU power usage: 300 W
Citation
Waktaverse-Llama-3
TBD
Llama-3
@article{llama3modelcard,
title={Llama 3 Model Card},
author={AI@Meta},
year={2024},
url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}
}
Model Card Authors
[More Information Needed]
Model Card Contact
[More Information Needed]
- Downloads last month
- 255