FinShibainu Model Card
- github: https://github.com/aiqwe/FinShibainu
- dataset: https://huggingface.co/datasets/aiqwe/FinShibainu
λͺ¨λΈμ KRX LLM κ²½μ§λν 리λ보λμμ μ°μμμ μμν shibainu24 λͺ¨λΈμ λλ€. λͺ¨λΈμ κΈμ΅, νκ³ λ± κΈμ΅κ΄λ ¨ μ§μμ λν Text Generationμ μ 곡ν©λλ€.
- Vanilla model : Qwen2.5-7B-Instruct
λ°μ΄ν°μ μμ§ λ° νμ΅μ κ΄λ ¨λ μ½λλ https://github.com/aiqwe/FinShibainuμ μμΈνκ² κ³΅κ°λμ΄ μμ΅λλ€.
Usage
https://github.com/aiqwe/FinShibainuμ exampleμ μ°Έμ‘°νλ©΄ μ½κ² inferenceλ₯Ό ν΄λ³Ό μ μμ΅λλ€. λλΆλΆμ Inferenceλ RTX-3090 μ΄μμμ λ¨μΌ GPU κ°λ₯ν©λλ€.
pip install vllm
import pandas as pd
from vllm import LLM
inputs = [
"μΈνμμ₯μμ μΌλ³Έ μνμ λ―Έκ΅ λ¬λ¬μ νμ¨μ΄ λ μμ₯μμ μ½κ°μ μ°¨μ΄λ₯Ό 보μ΄κ³ μλ€. μ΄λ 무μν μ΄μ΅μ μ»κΈ° μν μ μ ν κ±°λ μ λ΅μ 무μμΈκ°?",
"μ μ£ΌμΈμκΆλΆμ¬μ±(BW)μμ μ±κΆμκ° μ μ£ΌμΈμκΆμ νμ¬νμ§ μμ κ²½μ° μ΄λ€ μΌμ΄ λ°μνλκ°?",
"곡맀λ(Short Selling)μ λν μ€λͺ
μΌλ‘ μ³μ§ μμ κ²μ 무μμ
λκΉ?"
]
llm = LLM(model="aiqwe/krx-llm-competition", tensor_parallel_size=1)
sampling_params = SamplingParams(temperature=0.7, max_tokens=128)
outputs = llm.generate(inputs, sampling_params)
for o in outputs:
print(o.prompt)
print(o.outputs[0].text)
print("*"*100)
Model Card
Contents | Spec |
---|---|
Base model | Qwen2.5-7B-Instruct |
dtype | bfloat16 |
PEFT | LoRA (r=8, alpha=64) |
Learning Rate | 1e-5 (varies by further training) |
LRScheduler | Cosine (warm-up: 0.05%) |
Optimizer | AdamW |
Distributed / Efficient Tuning | DeepSpeed v3, Flash Attention |
Datset Card
Reference λ°μ΄ν°μ
μ μΌλΆ μ μκΆ κ΄κ³λ‘ μΈν΄ Linkλ‘ μ 곡ν©λλ€.
MCQAμ QA λ°μ΄ν°μ
μ https://huggingface.co/datasets/aiqwe/FinShibainuμΌλ‘ 곡κ°ν©λλ€.
λν https://github.com/aiqwe/FinShibainuλ₯Ό μ΄μ©νλ©΄ λ€μν μ νΈλ¦¬ν° κΈ°λ₯μ μ 곡νλ©°, λ°μ΄ν° μμ± Pipelineμ μ°Έμ‘°ν μ μμ΅λλ€.
References
λ°μ΄ν°λͺ | url |
---|---|
νκ΅μν κ²½μ κΈμ΅ μ©μ΄ 700μ | Link |
μ¬λ¬΄νκ³ ν©μ± λ°μ΄ν° | μ체 μ μ |
κΈμ΅κ°λ μ©μ΄μ¬μ | Link |
web-text.synthetic.dataset-50k | Link |
μ§μκ²½μ μ©μ΄μ¬μ | Link |
νκ΅κ±°λμ λΉμ κΈ° κ°νλ¬Ό | Link |
νκ΅κ±°λμκ·μ | Link |
μ΄λ³΄ν¬μμ μ¦κΆλ°λΌμ‘κΈ° | Link |
μ²μλ μ μν μ¦κΆν¬μ | Link |
κΈ°μ μ¬μ λ³΄κ³ μ 곡μμλ£ | Link |
μμ¬κ²½μ μ©μ΄μ¬μ | Link |
MCQA
MCQA λ°μ΄ν°λ Referenceλ₯Ό κΈ°λ°μΌλ‘ λ€μ§μ λ€ν λ¬Έμ λ₯Ό μμ±ν λ°μ΄ν°μ
μ
λλ€. λ¬Έμ μ λ΅ λΏλ§ μλλΌ Reasoning ν
μ€νΈκΉμ§ μμ±νμ¬ νμ΅μ μΆκ°νμμ΅λλ€.
νμ΅μ μ¬μ©λ λ°μ΄ν°λ μ½ 4.5λ§κ° λ°μ΄ν°μ
μ΄λ©°, tiktokenμ o200k_base(gpt-4o, gpt-4o-mini Tokenizer)λ₯Ό κΈ°μ€μΌλ‘ μ΄ 2μ²λ§κ°μ ν ν°μΌλ‘ νμ΅λμμ΅λλ€.
λ°μ΄ν°λͺ | λ°μ΄ν° μ | ν ν° μ |
---|---|---|
νκ΅μν κ²½μ κΈμ΅ μ©μ΄ 700μ | 1,203 | 277,114 |
μ¬λ¬΄νκ³ λͺ©μ°¨λ₯Ό μ΄μ©ν ν©μ±λ°μ΄ν° | 451 | 99,770 |
κΈμ΅κ°λ μ©μ΄μ¬μ | 827 | 214,297 |
hf_web_text_synthetic_dataset_50k | 25,461 | 7,563,529 |
μ§μκ²½μ μ©μ΄μ¬μ | 2,314 | 589,763 |
νκ΅κ±°λμ λΉμ κΈ° κ°νλ¬Ό | 1,183 | 230,148 |
νκ΅κ±°λμκ·μ | 3,015 | 580,556 |
μ΄λ³΄ν¬μμ μ¦κΆλ°λΌμ‘κΈ° | 599 | 116,472 |
μ²μλ μ μν μ¦κΆ ν¬μ | 408 | 77,037 |
κΈ°μ μ¬μ λ³΄κ³ μ 곡μμλ£ | 3,574 | 629,807 |
μμ¬κ²½μ μ©μ΄μ¬μ | 7,410 | 1,545,842 |
ν©κ³ | 46,445 | 19,998,931 |
QA
QA λ°μ΄ν°λ Referenceμ μ§λ¬Έμ ν¨κ» InputμΌλ‘ λ°μ μμ±ν λ΅λ³κ³Ό Reference μμ΄ μ§λ¬Έλ§μ InputμΌλ‘ λ°μ μμ±ν λ΅λ³ 2κ°μ§λ‘ ꡬμ±λ©λλ€.
Referenceλ₯Ό μ 곡λ°μΌλ©΄ λͺ¨λΈμ λ³΄λ€ μ νν λ΅λ³μ νμ§λ§ λͺ¨λΈλ§μ μ§μμ΄ μ νλμ΄ λ΅λ³μ΄ μ’λ 짧μμ§κ±°λ λ€μμ±μ΄ μ€μ΄λ€κ² λ©λλ€.
μ΄ 4.8λ§κ°μ λ°μ΄ν°μ
κ³Ό 2μ΅κ°μ ν ν°μΌλ‘ νμ΅λμμ΅λλ€.
λ°μ΄ν°λͺ | λ°μ΄ν° μ | ν ν° μ |
---|---|---|
νκ΅μν κ²½μ κΈμ΅ μ©μ΄ 700μ | 1,023 | 846,970 |
κΈμ΅κ°λ μ©μ΄μ¬μ | 4,128 | 3,181,831 |
μ§μκ²½μ μ©μ΄μ¬μ | 6,526 | 5,311,890 |
νκ΅κ±°λμ λΉμ κΈ° κ°νλ¬Ό | 1,510 | 1,089,342 |
νκ΅κ±°λμκ·μ | 4,858 | 3,587,059 |
κΈ°μ μ¬μ λ³΄κ³ μ 곡μμλ£ | 3,574 | 629,807 |
μμ¬κ²½μ μ©μ΄μ¬μ | 29,920 | 5,981,839 |
ν©κ³ | 47,965 | 199,998,931 |
Citation
@misc{jaylee2024finshibainu,
author = {Jay Lee},
title = {FinShibainu: Korean specified finance model},
year = {2024},
publisher = {GitHub},
journal = {GitHub repository},
url = {https://github.com/aiqwe/FinShibainu}
}
- Downloads last month
- 75