Hugging Face's logo

Fikaaw
/
en-setfit-absa-model-aspect

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
absa
generated_from_setfit_trainer
Eval Results
Model card Files Community
Edit model card

SetFit Aspect Model with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification. In particular, this model is in charge of filtering aspect span candidates.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

This model was trained within the context of a larger system for ABSA, which looks like so:

  1. Use a spaCy model to select possible aspect span candidates.
  2. Use this SetFit model to filter these possible aspect span candidates.
  3. Use a SetFit model to classify the filtered aspect span candidates.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
no aspect
  • 'food:The food is really delicious! The meat is tender and the spices are well seasoned. I will definitely come back again.'
  • 'meat:The food is really delicious! The meat is tender and the spices are well seasoned. I will definitely come back again.'
  • 'spices:The food is really delicious! The meat is tender and the spices are well seasoned. I will definitely come back again.'
aspect
  • 'Service:Service is standard, nothing extraordinary.'
  • 'Service:Service from the staff is very friendly.'
  • 'Service:Service from the staff is very fast and professional.'

Evaluation

Metrics

Label Accuracy
all 1.0

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import AbsaModel

# Download from the 🤗 Hub
model = AbsaModel.from_pretrained(
    "models/en-setfit-absa-model-aspect",
    "models/en-setfit-absa-model-polarity",
)
# Run inference
preds = model("The food was great, but the venue is just way too busy.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 4 14.3487 72
Label Training Sample Count
no aspect 1701
aspect 14

Training Hyperparameters

  • batch_size: (4, 4)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 20
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0001 1 0.34 -
0.0029 50 0.318 -
0.0058 100 0.2344 -
0.0087 150 0.1925 -
0.0117 200 0.1893 -
0.0146 250 0.014 -
0.0175 300 0.0017 -
0.0204 350 0.0041 -
0.0233 400 0.0008 -
0.0262 450 0.0008 -
0.0292 500 0.0003 -
0.0321 550 0.0003 -
0.0350 600 0.0004 -
0.0379 650 0.0004 -
0.0408 700 0.0004 -
0.0437 750 0.0008 -
0.0466 800 0.0004 -
0.0496 850 0.0002 -
0.0525 900 0.0003 -
0.0554 950 0.0001 -
0.0583 1000 0.0001 -
0.0612 1050 0.0002 -
0.0641 1100 0.0002 -
0.0671 1150 0.0002 -
0.0700 1200 0.0001 -
0.0729 1250 0.0002 -
0.0758 1300 0.0001 -
0.0787 1350 0.0 -
0.0816 1400 0.0001 -
0.0845 1450 0.0001 -
0.0875 1500 0.0001 -
0.0904 1550 0.0001 -
0.0933 1600 0.0001 -
0.0962 1650 0.0001 -
0.0991 1700 0.0 -
0.1020 1750 0.0001 -
0.1050 1800 0.0001 -
0.1079 1850 0.0001 -
0.1108 1900 0.0001 -
0.1137 1950 0.0 -
0.1166 2000 0.0001 -
0.1195 2050 0.0001 -
0.1224 2100 0.0 -
0.1254 2150 0.0006 -
0.1283 2200 0.0002 -
0.1312 2250 0.0 -
0.1341 2300 0.0 -
0.1370 2350 0.2106 -
0.1399 2400 0.0 -
0.1429 2450 0.0001 -
0.1458 2500 0.0001 -
0.1487 2550 0.0 -
0.1516 2600 0.0 -
0.1545 2650 0.0 -
0.1574 2700 0.0 -
0.1603 2750 0.0 -
0.1633 2800 0.0 -
0.1662 2850 0.0001 -
0.1691 2900 0.0 -
0.1720 2950 0.0 -
0.1749 3000 0.0 -
0.1778 3050 0.0001 -
0.1808 3100 0.0 -
0.1837 3150 0.0 -
0.1866 3200 0.0001 -
0.1895 3250 0.0 -
0.1924 3300 0.0001 -
0.1953 3350 0.0001 -
0.1983 3400 0.0 -
0.2012 3450 0.0 -
0.2041 3500 0.0 -
0.2070 3550 0.0 -
0.2099 3600 0.0 -
0.2128 3650 0.0 -
0.2157 3700 0.0 -
0.2187 3750 0.0 -
0.2216 3800 0.0 -
0.2245 3850 0.0 -
0.2274 3900 0.0 -
0.2303 3950 0.0 -
0.2332 4000 0.0 -
0.2362 4050 0.0 -
0.2391 4100 0.0 -
0.2420 4150 0.0 -
0.2449 4200 0.0 -
0.2478 4250 0.0 -
0.2507 4300 0.0 -
0.2536 4350 0.0 -
0.2566 4400 0.0 -
0.2595 4450 0.0 -
0.2624 4500 0.0 -
0.2653 4550 0.0 -
0.2682 4600 0.0 -
0.2711 4650 0.0 -
0.2741 4700 0.0001 -
0.2770 4750 0.0 -
0.2799 4800 0.0 -
0.2828 4850 0.0 -
0.2857 4900 0.0 -
0.2886 4950 0.0 -
0.2915 5000 0.0 -
0.2945 5050 0.0 -
0.2974 5100 0.0 -
0.3003 5150 0.0 -
0.3032 5200 0.0 -
0.3061 5250 0.0 -
0.3090 5300 0.0 -
0.3120 5350 0.0 -
0.3149 5400 0.0 -
0.3178 5450 0.0 -
0.3207 5500 0.0 -
0.3236 5550 0.0 -
0.3265 5600 0.0 -
0.3294 5650 0.0 -
0.3324 5700 0.0 -
0.3353 5750 0.0 -
0.3382 5800 0.0 -
0.3411 5850 0.0 -
0.3440 5900 0.0 -
0.3469 5950 0.0 -
0.3499 6000 0.0 -
0.3528 6050 0.0 -
0.3557 6100 0.0 -
0.3586 6150 0.0 -
0.3615 6200 0.0 -
0.3644 6250 0.0 -
0.3673 6300 0.0 -
0.3703 6350 0.0 -
0.3732 6400 0.0001 -
0.3761 6450 0.0 -
0.3790 6500 0.0 -
0.3819 6550 0.0 -
0.3848 6600 0.0 -
0.3878 6650 0.0 -
0.3907 6700 0.0 -
0.3936 6750 0.0 -
0.3965 6800 0.0 -
0.3994 6850 0.0 -
0.4023 6900 0.0 -
0.4052 6950 0.0 -
0.4082 7000 0.0 -
0.4111 7050 0.0 -
0.4140 7100 0.0001 -
0.4169 7150 0.0 -
0.4198 7200 0.0 -
0.4227 7250 0.0 -
0.4257 7300 0.0 -
0.4286 7350 0.0 -
0.4315 7400 0.0 -
0.4344 7450 0.0 -
0.4373 7500 0.0 -
0.4402 7550 0.0 -
0.4431 7600 0.0 -
0.4461 7650 0.0 -
0.4490 7700 0.0 -
0.4519 7750 0.0 -
0.4548 7800 0.0 -
0.4577 7850 0.0 -
0.4606 7900 0.0 -
0.4636 7950 0.0 -
0.4665 8000 0.0 -
0.4694 8050 0.0 -
0.4723 8100 0.0 -
0.4752 8150 0.0 -
0.4781 8200 0.0 -
0.4810 8250 0.0 -
0.4840 8300 0.0 -
0.4869 8350 0.0001 -
0.4898 8400 0.0 -
0.4927 8450 0.0 -
0.4956 8500 0.0 -
0.4985 8550 0.0 -
0.5015 8600 0.0 -
0.5044 8650 0.0 -
0.5073 8700 0.0 -
0.5102 8750 0.0 -
0.5131 8800 0.0 -
0.5160 8850 0.0 -
0.5190 8900 0.0 -
0.5219 8950 0.0 -
0.5248 9000 0.0 -
0.5277 9050 0.0 -
0.5306 9100 0.0 -
0.5335 9150 0.0 -
0.5364 9200 0.0 -
0.5394 9250 0.0 -
0.5423 9300 0.0 -
0.5452 9350 0.0 -
0.5481 9400 0.0 -
0.5510 9450 0.0 -
0.5539 9500 0.0 -
0.5569 9550 0.0 -
0.5598 9600 0.0 -
0.5627 9650 0.0 -
0.5656 9700 0.0 -
0.5685 9750 0.0 -
0.5714 9800 0.0 -
0.5743 9850 0.0 -
0.5773 9900 0.0 -
0.5802 9950 0.0 -
0.5831 10000 0.0 -
0.5860 10050 0.0 -
0.5889 10100 0.0 -
0.5918 10150 0.0 -
0.5948 10200 0.0 -
0.5977 10250 0.0 -
0.6006 10300 0.0 -
0.6035 10350 0.0 -
0.6064 10400 0.0 -
0.6093 10450 0.0 -
0.6122 10500 0.0 -
0.6152 10550 0.0 -
0.6181 10600 0.0 -
0.6210 10650 0.0 -
0.6239 10700 0.0 -
0.6268 10750 0.0 -
0.6297 10800 0.0 -
0.6327 10850 0.0 -
0.6356 10900 0.0 -
0.6385 10950 0.0 -
0.6414 11000 0.0 -
0.6443 11050 0.0 -
0.6472 11100 0.0 -
0.6501 11150 0.0 -
0.6531 11200 0.0 -
0.6560 11250 0.0 -
0.6589 11300 0.0 -
0.6618 11350 0.0 -
0.6647 11400 0.0 -
0.6676 11450 0.0 -
0.6706 11500 0.0 -
0.6735 11550 0.0 -
0.6764 11600 0.0 -
0.6793 11650 0.0 -
0.6822 11700 0.0 -
0.6851 11750 0.0 -
0.6880 11800 0.0 -
0.6910 11850 0.0 -
0.6939 11900 0.0 -
0.6968 11950 0.0 -
0.6997 12000 0.0 -
0.7026 12050 0.0 -
0.7055 12100 0.0 -
0.7085 12150 0.0 -
0.7114 12200 0.0 -
0.7143 12250 0.0 -
0.7172 12300 0.0 -
0.7201 12350 0.0 -
0.7230 12400 0.0 -
0.7259 12450 0.0 -
0.7289 12500 0.0 -
0.7318 12550 0.0 -
0.7347 12600 0.0 -
0.7376 12650 0.0 -
0.7405 12700 0.0 -
0.7434 12750 0.0 -
0.7464 12800 0.0 -
0.7493 12850 0.0 -
0.7522 12900 0.0 -
0.7551 12950 0.0 -
0.7580 13000 0.0 -
0.7609 13050 0.0 -
0.7638 13100 0.0 -
0.7668 13150 0.0 -
0.7697 13200 0.0 -
0.7726 13250 0.0 -
0.7755 13300 0.0 -
0.7784 13350 0.0 -
0.7813 13400 0.0 -
0.7843 13450 0.0 -
0.7872 13500 0.0 -
0.7901 13550 0.0 -
0.7930 13600 0.0 -
0.7959 13650 0.0 -
0.7988 13700 0.0 -
0.8017 13750 0.0 -
0.8047 13800 0.0 -
0.8076 13850 0.0 -
0.8105 13900 0.0 -
0.8134 13950 0.0 -
0.8163 14000 0.0 -
0.8192 14050 0.0 -
0.8222 14100 0.0 -
0.8251 14150 0.0 -
0.8280 14200 0.0 -
0.8309 14250 0.0 -
0.8338 14300 0.0 -
0.8367 14350 0.0 -
0.8397 14400 0.0 -
0.8426 14450 0.0 -
0.8455 14500 0.0 -
0.8484 14550 0.0 -
0.8513 14600 0.0 -
0.8542 14650 0.0 -
0.8571 14700 0.0 -
0.8601 14750 0.0 -
0.8630 14800 0.0 -
0.8659 14850 0.0 -
0.8688 14900 0.0 -
0.8717 14950 0.0 -
0.8746 15000 0.0 -
0.8776 15050 0.0 -
0.8805 15100 0.0 -
0.8834 15150 0.0 -
0.8863 15200 0.0 -
0.8892 15250 0.0 -
0.8921 15300 0.0 -
0.8950 15350 0.0 -
0.8980 15400 0.0 -
0.9009 15450 0.0 -
0.9038 15500 0.0 -
0.9067 15550 0.0 -
0.9096 15600 0.0 -
0.9125 15650 0.0 -
0.9155 15700 0.0 -
0.9184 15750 0.0 -
0.9213 15800 0.0 -
0.9242 15850 0.0 -
0.9271 15900 0.0 -
0.9300 15950 0.0 -
0.9329 16000 0.0 -
0.9359 16050 0.0 -
0.9388 16100 0.0 -
0.9417 16150 0.0 -
0.9446 16200 0.0 -
0.9475 16250 0.0 -
0.9504 16300 0.0 -
0.9534 16350 0.0 -
0.9563 16400 0.0 -
0.9592 16450 0.0 -
0.9621 16500 0.0 -
0.9650 16550 0.0 -
0.9679 16600 0.0 -
0.9708 16650 0.0 -
0.9738 16700 0.0 -
0.9767 16750 0.0 -
0.9796 16800 0.0 -
0.9825 16850 0.0 -
0.9854 16900 0.0 -
0.9883 16950 0.0 -
0.9913 17000 0.0 -
0.9942 17050 0.0 -
0.9971 17100 0.0 -
1.0 17150 0.0 -

Framework Versions

  • Python: 3.10.13
  • SetFit: 1.0.3
  • Sentence Transformers: 2.7.0
  • spaCy: 3.7.4
  • Transformers: 4.39.3
  • PyTorch: 2.1.2
  • Datasets: 2.18.0
  • Tokenizers: 0.15.2

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
9
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
Text Classification
Inference API (serverless) has been turned off for this model.

Finetuned from

Evaluation results

View on Papers With Code