yt-sentiment-roberta

A sentiment classifier for English YouTube comments. It labels each comment as negative, neutral or positive. It is a fine-tuned version of cardiffnlp/twitter-roberta-base-sentiment-latest, trained on a cleaned English subset of AmaanP314/youtube-comment-sentiment.

Model Details

• Developed by: roshan050
• Model type: RoBERTa-base sequence classification (3 classes: negative, neutral, positive)
• Language: English
• License: CC BY-SA 4.0 (the base model is CC BY 4.0 and the training dataset is CC BY-SA 4.0)
• Fine-tuned from: cardiffnlp/twitter-roberta-base-sentiment-latest
• Demo / API: roshan050/sentiment-analysis-gradio

Uses

Direct use

Classifying the sentiment of English YouTube comments.

Downstream use

It is the sentiment backend of the Yt_Analyzer Flutter app (YouTube channel analytics).

Out-of-scope use

• Languages other than English, including Hinglish and Hindi.
• Other platforms and text types; it was only trained and tested on YouTube comments.
• Toxicity, hate speech or abuse detection, and decisions about individual people.

Bias, Risks, and Limitations

• The training and test labels were produced by an AI model with some manual validation, not by a fully human-labelled process. The scores below measure agreement with those labels.
• Neutral is the hardest class (F1 0.6706). Sarcasm and mixed sentiment are often misclassified.
• Results come from one dataset. The model was not evaluated for fairness across topics, groups or time periods, and it can reflect biases in the source data and the labelling model.

How to Get Started

Replace links and user mentions the same way as in training, then run the model:

import re
from transformers import pipeline

clf = pipeline("text-classification", model="roshan050/yt-sentiment-roberta",
               top_k=None, truncation=True, max_length=96)

def prep(t):
    t = re.sub(r"https?://\S+", "http", str(t))
    t = re.sub(r"@[\w.\-]+", "@user", t)
    return re.sub(r"\s+", " ", t).strip()

print(clf([prep("great video, loved it!")]))

Training Details

Training data

AmaanP314/youtube-comment-sentiment, cleaned as follows:

• Synthetic rows (comment IDs ending in "_aug") removed.
• Rows with missing text or label removed, duplicate comment texts removed, and very short comments (1 character) removed.
• Comments written mostly in non-Latin scripts removed, then English-only comments kept using language detection (langdetect).
• Videos were split first: 5% of videos were set aside for testing, so the test comments come from videos that never appear in training.
• Up to 20,000 comments per class were sampled for training from the remaining videos, and up to 6,000 comments for testing from the held-out videos.
• Final sizes after the English filter: 49,114 training, 2,585 validation (a 5% stratified split of the training sample), 5,186 test comments.

Preprocessing

URLs are replaced with "http" and user mentions with "@user". Maximum sequence length is 96 tokens.

Training hyperparameters

• Epochs: 2
• Learning rate: 2e-5
• Train batch size: 32 (evaluation batch size 128)
• Weight decay: 0.01
• Warmup steps: 300
• Mixed precision (fp16)
• Best checkpoint chosen by validation macro-F1
• Framework: Hugging Face Transformers Trainer, 3,070 steps

Speeds, sizes, times

About 8.5 minutes of training on a Google Colab GPU.

Validation after each epoch: accuracy 0.7532 and macro-F1 0.7544 after epoch 1; accuracy 0.7621 and macro-F1 0.7587 after epoch 2.

Evaluation

Testing data

5,186 English comments (1,580 negative, 1,503 neutral, 2,103 positive) from videos not used for training or validation.

Results

Model	Accuracy	Macro precision	Macro recall	Macro F1
VADER (lexicon baseline)	0.5605	0.5503	0.5346	0.5329
cardiffnlp/twitter-roberta-base-sentiment-latest (no fine-tuning)	0.6996	0.6923	0.6939	0.6902
yt-sentiment-roberta (this model)	0.7601	0.7523	0.7531	0.7511

Per class (this model):

Class	Precision	Recall	F1	Support
negative	0.7122	0.8114	0.7586	1580
neutral	0.7018	0.6420	0.6706	1503
positive	0.8429	0.8060	0.8240	2103

Confusion matrix (rows are true labels, columns are predictions):

True \ Predicted	negative	neutral	positive
negative	1282	209	89
neutral	311	965	227
positive	207	201	1695

Environmental Impact

• Hardware: Google Colab GPU (free tier; exact GPU type not recorded)
• Training time: about 8.5 minutes
• Carbon emissions: not measured

Citation

Base model: cardiffnlp/twitter-roberta-base-sentiment-latest (paper: arXiv:2202.03829).

Dataset: Amaan Poonawala (2025). YouTube Comment Sentiment Dataset (1M+ labeled comments) [Data set]. Hugging Face. https://huggingface.co/datasets/AmaanP314/youtube-comment-sentiment

Model Card Contact

Hugging Face user: roshan050