Steeve2ml/globatrend-absa-english-classifier

Model Card for globatrend-absa-classifier

Model Details

Model Description

DistilBERT fine-tuned for Aspect-Based Sentiment Analysis (ABSA) — given a review and a specific aspect mentioned in it (e.g. "delivery", "customer service"), predicts the sentiment (negative/neutral/positive) for that aspect specifically, rather than one sentiment for the whole review. Built as part of GlobaTrend Insights, a portfolio project covering the full modern NLP pipeline. This is the project's flagship model (Phase 9), the one that answers the core business question the project was built around: not just "is the customer happy?" but "happy or unhappy about what?"

• Developed by: Independent portfolio project (GlobaTrend Insights)
• Model type: Transformer encoder (DistilBERT), sentence-pair sequence classification
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: distilbert-base-uncased

Model Sources

• Base model: https://huggingface.co/distilbert-base-uncased
• Training data: SemEval-2014 Task 4 (restaurant reviews), via https://huggingface.co/datasets/tomaarsen/setfit-absa-semeval-restaurants
• Related model: globatrend-sentiment-distilbert (whole-review binary sentiment, no aspect breakdown) — same project, Phase 6

Uses

Direct Use

Given an English review and one or more aspect terms found in it (e.g. "the delivery was fast but the price was high" + ["delivery", "price"]), classifies the sentiment for each aspect independently. Intended for e-commerce and restaurant-style customer reviews.

Input format: a sentence pair — the full review text, and the aspect term as a short phrase — tokenized jointly (tokenizer(text, aspect)), following the standard sentence-pair classification convention.

Downstream Use

Intended to be paired with an aspect-extraction step (this project uses a POS-tagging heuristic, see Limitations) to build a full pipeline: raw review in, {aspect: sentiment} breakdown out.

Out-of-Scope Use

• Not multilingual — trained and evaluated on English only. A separate experimental XLM-R model (not this one) was fine-tuned on the same English data and evaluated zero-shot on Spanish, German, French, and Hindi as part of this project's research question, but its results are unstable and it is not published as a production-ready model.
• Not an aspect extractor — this model does not find aspects in text by itself; it only classifies sentiment once an aspect is given. Aspect discovery is a separate step.
• Not validated on long, multi-clause reviews — see Limitations. Works best on reviews focused on one or two aspects per sentence.
• Not intended for high-stakes decisions without further validation on your own domain.

Bias, Risks, and Limitations

• Degrades on long, multi-aspect reviews (verified empirically): on a four-clause review ("delivery was fast and product quality is excellent, but customer service was rude and the refund process took forever"), the model correctly classified the two negative aspects but incorrectly predicted negative for both positive aspects too. On the same clauses tested in isolation, prediction quality was noticeably better. This suggests the model can be influenced by the overall tone of a long review rather than correctly isolating the segment relevant to each aspect. Shorter, more focused reviews are classified more reliably.
• "Conflict" polarity folded into "neutral": the source dataset (SemEval-2014) includes a fourth polarity, "conflict" (an aspect praised and criticized in the same sentence), which this model maps to "neutral" during training rather than treating as a separate class. Genuinely mixed opinions on a single aspect may therefore be reported as neutral rather than flagged as conflicting.
• Aspect extraction (if paired with this model) is English-only and heuristic: the companion aspect-extraction step used in this project relies on English part-of-speech tagging and does not generalize to other languages or domains without adaptation.
• Domain: trained on restaurant reviews (SemEval-2014); may transfer imperfectly to other e-commerce categories without further fine-tuning.

Recommendations

For reviews with several aspects and shifting sentiment across clauses, consider a pipeline that first splits the review into shorter segments before classifying each aspect, rather than passing the full multi-clause review directly. Validate on your own domain before production use, and treat "neutral" predictions on strongly-worded text as a possible signal of a mixed (praised-and- criticized) opinion rather than genuine indifference.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/globatrend-absa-classifier")
model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/globatrend-absa-classifier")

text = "The delivery was fast but the price was too high"
aspect = "delivery"

inputs = tokenizer(text, aspect, return_tensors="pt", truncation=True)
with torch.no_grad():
    logits = model(**inputs).logits
predicted_class = torch.argmax(logits, dim=-1).item()
print(model.config.id2label[predicted_class])

Training Details

Training Data

SemEval-2014 Task 4 (restaurant reviews), using the dataset's native train split (aspect-level polarity labels: negative, neutral, positive, conflict — conflict remapped to neutral). The dataset's native test split had no usable polarity labels for this task, so evaluation data was instead held out from the train split (80/20).

Training Procedure

Preprocessing

Tokenized as a sentence pair: tokenizer(review_text, aspect_term), inserting [SEP] between the two segments with token_type_ids distinguishing them, truncation and padding enabled.

Training Hyperparameters

• Base model: distilbert-base-uncased (~67M parameters)
• Epochs: 5
• Learning rate: 2e-5
• Batch size: 16
• Optimizer: AdamW (Hugging Face Trainer default)
• Training regime: fp32
• Classes: 3 (negative, neutral, positive)

Evaluation

Testing Data

Held-out 20% split from the SemEval-2014 Task 4 restaurant train data (native test split unusable for polarity evaluation, see Training Data).

Metrics

Accuracy and macro F1-score (averaged across the three polarity classes, appropriate for multi-class rather than binary classification).

Results

Metric	Value
Eval loss	0.573
Accuracy	0.767
F1 (macro)	0.698

Summary

The model reaches 76.7% accuracy classifying aspect-level sentiment on held-out restaurant reviews, a substantial improvement over an earlier iteration trained on a small hand-written example set (which collapsed to predicting a single class regardless of input). It performs noticeably better on short, single-aspect statements than on long reviews covering several aspects with shifting sentiment — see Limitations for a concrete example.