poseidon1113/gpt2-lora-financial-sentiment-v1

GPT-2 LoRA — Financial Sentiment Analysis

LoRA fine-tuned GPT-2 for financial sentiment classification (positive / neutral / negative). Trained on Financial PhraseBank sentences_allagree split.

Model Details

Base model	GPT-2 (124M)
Method	LoRA (r=8, alpha=16)
Trainable params	~300K (0.24%)
Target modules	c_attn
Dataset	Financial PhraseBank — 4,840 examples
Epochs	3
Max length	128
Learning rate	2e-4

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

tokenizer = AutoTokenizer.from_pretrained("poseidon1113/gpt2-lora-financial-sentiment-v1")
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained("gpt2", torch_dtype=torch.float16),
    "poseidon1113/gpt2-lora-financial-sentiment-v1"
).eval()

def predict(sentence):
    inputs = tokenizer(f"### Sentence:\n{sentence}\n\n### Sentiment:\n", return_tensors="pt")
    with torch.no_grad():
        out = model.generate(**inputs, max_new_tokens=10, do_sample=False, pad_token_id=tokenizer.eos_token_id)
    result = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip().lower()
    return next((w for w in result.split() if w in ("positive", "negative", "neutral")), "neutral")

predict("Operating profit rose to EUR 13.1 mn from EUR 21.1 mn.")  # → positive

Evaluation on FiQA 2018 (all splits)

Class	Correct	Total	Accuracy
Positive	66	706	9.3%
Neutral	108	115	93.9%
Negative	16	373	4.3%
Overall	190	1194	15.9%

Limitations

• Trained on formal European financial news — may underperform on social media / cashtag text
• Neutral class slightly over-predicted due to training distribution (~60% neutral in PhraseBank)

Citation

@article{Malo2014GoodDO,
  title={Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts},
  author={P. Malo and A. Sinha and P. Korhonen and J. Wallenius and P. Virtanen},
  journal={Journal of the Association for Information Science and Technology},
  year={2014}, volume={65}
}