---
license: apache-2.0
base_model: Qwen/Qwen1.5-0.5B-Chat
tags:
- trl
- orpo
model-index:
- name: Qwen-Orpo-v1
  results: []
---

## FINGU-AI/Qwen-Orpo-v1

### Overview

The FINGU-AI/Qwen-Orpo-v1 model offers a specialized curriculum tailored to English, speakers interested in finance, investment, and legal frameworks. It aims to enhance language proficiency while providing insights into global finance markets and regulatory landscapes.

### Key Features

- **Global Perspective**: Explores diverse financial markets and regulations across English, Korean, and Japanese contexts.
- **Language Proficiency**: Enhances language skills in English, Korean, and Japanese for effective communication in finance and legal domains.
- **Career Advancement**: Equips learners with knowledge and skills for roles in investment banking, corporate finance, asset management, and regulatory compliance.

### Model Information

- **Model Name**: FINGU-AI/Qwen-Orpo-v1
- **Description**: FINGU-AI/Qwen-Orpo-v1 model trained on various languages, including English.
- **Checkpoint**: FINGU-AI/Qwen-Orpo-v1
- **Author**: Grinda AI Inc.
- **License**: Apache-2.0

### Training Details

- **Fine-Tuning**: The model was fine-tuned on the base model Qwen/Qwen1.5-0.5B-Chat through ORPO fine-tuning using the TrL Library and Transformer.
- **Dataset**: The fine-tuning dataset consisted of 28k training samples.


### How to Use

To use the FINGU-AI/Qwen-Orpo-v1 model, you can utilize the Hugging Face Transformers library.
Here's a Python code snippet demonstrating how to load the model and generate predictions:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig,TextStreamer


model_id = 'FINGU-AI/Qwen-Orpo-v1'
model = AutoModelForCausalLM.from_pretrained(model_id, attn_implementation="flash_attention_2", torch_dtype= torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(tokenizer)
model.to('cuda')



messages = [
    {"role": "system","content": " you are as a finance specialist, help the user and provide accurat information."},
    {"role": "user", "content": " what are the best approch to prevent loss?"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")

generation_params = {
    'max_new_tokens': 1000,
    'use_cache': True,
    'do_sample': True,
    'temperature': 0.7,
    'top_p': 0.9,
    'top_k': 50,
}

outputs = model.generate(tokenized_chat, **generation_params, streamer=streamer)
decoded_outputs = tokenizer.batch_decode(outputs)

```