adaption-nihongo-legal-finance-qa

This repository contains LoRA adapter weights trained with Adaption AutoScientist for Japanese legal and finance expert QA.

Intended Use

The adapter is intended for Japanese instruction following where the assistant explains legal and financial concepts, preserves uncertainty, avoids fabricated authority, and separates general explanation from professional advice.

Base Model

• Base model: mistralai/Mixtral-8x7B-Instruct-v0.1
• Training method: SFT
• Adapter type: LoRA
• Data format: chat

Training Data

• Source dataset: https://huggingface.co/datasets/doraking/nihongo-legal-finance-autoscientist-data
• Adapted dataset: https://huggingface.co/datasets/doraking/adaption-nihongo-legal-finance-qa
• Kaggle source dataset: https://www.kaggle.com/datasets/dorakingx/nihongo-legal-finance-autoscientist
• Kaggle adapted/model artifact: https://www.kaggle.com/datasets/dorakingx/adaption-nihongo-legal-finance-qa

Training Hyperparameters

• LoRA rank: 64
• LoRA alpha: 128
• LoRA dropout: 0.05
• Epochs: 4
• Learning rate: 0.00015
• Scheduler: cosine
• Warmup ratio: 0.03
• Weight decay: 0.01
• Max grad norm: 1
• Trainable modules: all-linear

How To Load

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "mistralai/Mixtral-8x7B-Instruct-v0.1"
adapter = "doraking/adaption-nihongo-legal-finance-qa"

tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

Limitations

This adapter is for educational and benchmark use. Legal and financial outputs should not be treated as professional legal, financial, tax, or investment advice. Users should verify high-stakes answers with qualified professionals.