davidkim205/exchange-8b

The exchange-8b model is an exchange rate prediction model that maximizes exchange rate prediction performance by applying Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) technologies to Meta's Llama 3.1 (8B) architecture. The exchange-8b model analyzes factors affecting exchange rates based on Prompt and Foreign Exchange Market-Related Data (exchange rate data, key economic indicator data, news data) and predicts the exchange rate for the next day. It predicts the USD/KRW exchange rate and outputs the exchange rate prediction results in CSV format. What makes this model different is that it is built on LLM (Large Language Model), which allows it to provide insight into the future of the uncertain forex market. In addition, it effectively processes and analyzes complex and diverse financial data, while integrating various unstructured data sources in real time to reflect the ever-changing foreign exchange market situation, enabling faster and more accurate exchange rate predictions than the existing method of direct human analysis and prediction.

Model Details

• Base Model: meta-llama/Llama-3.1-8B-Instruct
• Base Model Release Date: July 23, 2024
• Context Length: 128k
• License: llama 3.1
• Model Type: Text Generation
• Fine-Tuning Techniques
  • SFT (Supervised Fine-Tuning) Trained on diverse market data to improve the model's predictive accuracy, allowing for a more precise reflection of exchange rate fluctuation patterns.
  • DPO (Direct Preference Optimization) Optimized the model's predictions based on user preferences to enhance usability in practical scenarios.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("davidkim205/exchange-8b")
model = AutoModelForCausalLM.from_pretrained(
    "davidkim205/exchange-8b",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

text="""{주어진 다양한 금융 데이터와 뉴스 내용을 통해 24시간 USD 환율 데이터를 예측하시오. CSV 형식으로 Date,Name,Open,High,Low,Close으로 예측 결과를 제공하시오.

## USD Data
Date,Name,Open,High,Low,Close
2025-02-25,USD,1428.4,1435.0,1426.7,1433.1
2025-02-26,USD,1433.1,1436.4,1428.9,1436.3
2025-02-27,USD,1436.3,1447.3,1432.9,1446.3

## DOW Data
Date,Name,Open,High,Low,Close
2025-02-25,DOW,43467.0,43734.4,43302.6,43313.6
2025-02-26,DOW,43313.6,43858.7,43283.0,43813.8
2025-02-27,DOW,43813.8,43879.9,43318.9,43809.3

## KOSDAQ Data
Date,Name,Open,High,Low,Close
2025-02-25,KOSDAQ,766.8,773.9,765.3,769.0
2025-02-26,KOSDAQ,769.0,774.2,766.7,774.1
2025-02-27,KOSDAQ,774.1,776.6,760.1,760.1

## 2025-02-27 News
1. **EUR/USD**: 유로/달러 환율은 1.0500 아래에서 유지되고 있으며, 트럼프의 EU 관세 부과는 예상된 일이었다.  
2. **CNY/USD (위안화)**: 중국 중앙은행(PBoC)은 위안화 환율을 달러당 7.1740으로 조정.  
3. **GBP/USD (영국 파운드)**: 파운드/달러 환율은 1.2685를 기록하며 2025년 최고 수준에 도달했으나, 조정 가능성이 있다.  
4. **코스피 및 원/달러 환율 동향** - 코스피 지수 0.73% 하락, 원/달러 환율 0.67% 상승하며 1443.0원에 마감.  
5. **트럼프 관세 발언으로 인한 환율 변동** - 트럼프의 EU 관세 부과 발언으로 달러 강세, 원/달러 환율 9.9원 상승.  
}"""

messages = [
    {"role": "user", "content": text},
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda")

outputs = model.generate(**input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))

Date,Name,Open,High,Low,Close
2025-02-28,USD,1446.3,1463.8,1445.4,1461.8

Evaluation

Ranking

Ranking refers to the process of ordering multiple values to assess their relative magnitude. The following table ranks the accuracy of currency predictions, calculated based on the Mean Absolute Percentage Error (MAPE) between the predicted High and Low values from various banks and models and the actual USD exchange rates for March 2025. MAPE provides a standardized measure of error, where lower values indicate higher prediction accuracy.

To evaluate the model’s performance under different data conditions, we compared results using exchange rate data from Yahoo Finance, KEB Hana Bank, and SMBS (Seoul Money Brokerage Services). The Average column represents the overall average MAPE across all data sources(Yahoo Finance, KEB Hana Bank, SMBS). The Yahoo Finance, KEB Hana Bank and SMBS columns represent the average MAPE for High and Low values calculated using each respective data source.

Rank	Site	Average	Yahoo Finance	KEB Hana Bank	SMBS
1	Korea Trade Insurance Corp.	0.18	0.17	0.19	0.18
2	exchange-12b	0.19	0.17	0.21	0.20
3	Shinhan Bank	0.20	0.19	0.20	0.20
4	iM Bank	0.21	0.20	0.23	0.21
4	KOOKMIN BANK	0.21	0.19	0.23	0.21
4	Woori Bank	0.21	0.19	0.23	0.21
7	exchange-8b	0.22	0.21	0.24	0.22
8	exchange-3b	0.23	0.23	0.23	0.23
8	KEB Hana Bank	0.23	0.22	0.23	0.24
8	Toss Securities	0.23	0.22	0.24	0.23
11	exchange-1b	0.25	0.25	0.26	0.24

More detailed Evaluation Results can be found in the Technical Report.