FinTwitRoBERTa-wsb-sentiment

FinTwitRoBERTa-wsb-sentiment is a fine-tuned model for classifying the sentiment of r/wallstreetbets posts and comments as bullish, bearish or neutral. It uses FinTwitRoBERTa-sentiment as a base model and was fine-tuned on Reddit posts, because the slang, sarcasm and longer texts of retail trading subreddits differ from tweets. It is the successor of FinTwitBERT-wsb-sentiment. For tweets, use FinTwitRoBERTa-sentiment.

Intended Uses

FinTwitRoBERTa-wsb-sentiment is intended for classifying posts and comments on r/wallstreetbets and similar retail trading subreddits. Posts up to 256 tokens are read as a whole (about 97% of WSB posts); longer ones are cut off unless you split them.

Dataset

FinTwitRoBERTa-wsb-sentiment has been trained on:

• WSBS WSB_all_agree.csv: 1,509 r/wallstreetbets posts and comments that all annotators labeled the same, split 70/15/15 into training, validation and test posts exactly as for FinTwitBERT-wsb-sentiment.

In total 1,056 texts for training and 226 WSB posts for validation. The test posts were never used for training or for choosing the recipe.

Model Details

• Base model: FinTwitRoBERTa-sentiment.
• Labels: NEUTRAL, BULLISH, BEARISH.
• Training: 4 epochs, learning rate 2e-05 with 10% warmup and layer-wise learning-rate decay of 0.9, weight decay 0.01, batch size 16, maximum length 256 tokens, seed 42. The checkpoint with the best validation F1 (226 WSB posts) is kept. The recipe (from-sentiment-lr2e-05 in experiments_wsb.json) was chosen by its macro F1 on the WSB validation posts.
• Calibration: config.calibration_temperature = 2.011, fitted on the WSB validation posts. Divide the logits by it before the softmax to get confidence scores that match how often the model is right.

Evaluation

The test set is the 227 held-out posts of the WSBS split that FinTwitBERT-wsb-sentiment was also tested on, so the scores compare directly: FinTwitBERT-wsb-sentiment scores a macro F1 of 0.821 and FinTwitRoBERTa-sentiment (no WSB training) 0.657. The 95% bootstrap interval of the test macro F1 is 0.838–0.924: with 227 posts, differences of a few points are noise. The tweet test set (1,237 tweets from a financial Twitter feed) shows how well the model still reads tweets.

Metric	WSB test (227 posts)	WSB validation (226 posts)	Tweet test (1,237 tweets)
Macro F1	0.886	0.831	0.735
Accuracy	0.894	0.854	0.742
Bullish precision	0.928	0.893	0.658
Bullish recall	0.918	0.939	0.966
Bearish precision	0.895	0.825	0.553
Bearish recall	0.907	0.892	0.948
Neutral recall	0.833	0.648	0.575
Calibration error (ECE)	0.012	–	–

More Information

For a comprehensive overview, including the training setup and analysis of the model, visit the ModernFinTwitBERT GitHub repository.

Usage

Using HuggingFace's transformers library the model and tokenizers can be converted into a pipeline for text classification.

from transformers import pipeline

# Create a sentiment analysis pipeline
pipe = pipeline(
    "sentiment-analysis",
    model="StephanAkkerman/FinTwitRoBERTa-wsb-sentiment",
)

# Get the predicted sentiment
print(pipe("Come on TSLA. Fly you regarded bird!!"))

The model was trained with links replaced by [URL], user mentions (u/name) by @USER and Reddit emote markup removed, so do the same for the best results. The pipeline's scores are not calibrated; for confidence scores, divide the logits by config.calibration_temperature:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("StephanAkkerman/FinTwitRoBERTa-wsb-sentiment")
model = AutoModelForSequenceClassification.from_pretrained("StephanAkkerman/FinTwitRoBERTa-wsb-sentiment").eval()

inputs = tokenizer(["GME to the moon, holding my calls"], return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**inputs).logits
probs = (logits / model.config.calibration_temperature).softmax(-1)
print({model.config.id2label[i]: p for i, p in enumerate(probs[0].tolist())})

Citing & Authors

If you use FinTwitRoBERTa-wsb-sentiment in your research, please cite me as follows:

@misc{FinTwitRoBERTawsbsentiment,
  author = {Stephan Akkerman},
  title = {FinTwitRoBERTa-wsb-sentiment: A Sentiment Classifier for r/wallstreetbets},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/StephanAkkerman/FinTwitRoBERTa-wsb-sentiment}}
}

The training data is the WSBS dataset by moonscape95.

License

This project is licensed under the MIT License. See the LICENSE file for details.