Hugging Face's logo

pahri
/
setfit-indo-resto-RM-ibu-imas-aspect

Text Classification
setfit
Safetensors
sentence-transformers
bert
absa
generated_from_setfit_trainer
Eval Results
Model card Files Community
setfit-indo-resto-RM-ibu-imas-aspect / README.md
pahri's picture
pahri
Add SetFit ABSA model
bbdb9e5 verified
preview code
|
raw history blame contribute delete
No virus
33.5 kB
metadata
library_name: setfit
tags:
  - setfit
  - absa
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: >-
      Suasana:Tempatnya ramai sekali dan ngantei banget. Suasana di dalam resto
      sangat panas dan padat. Makanannya enak enak.
  - text: >-
      bener2 pedes puolll:Rasanya sgt gak cocok dilidah gue orang bekasi..
      ayamnya ayam kampung sih tp kecil bgt (beli yg dada).. terus tempe bacem
      sgt padet dan tahunya enak sih.. untuk sambel pedes bgt bener2 pedes
      puolll, tp rasanya gasukaa.
  - text: >-
      gang:Suasana di dalam resto sangat panas dan padat. Makanannya enak enak.
      Dan restonya ada di beberapa tempat dalam satu gang.
  - text: >-
      tempe:Menu makanannya khas Sunda ada ayam, pepes ikan, babat, tahu, tempe,
      sayur-sayur. Tidak banyak variasinya tapi kualitas rasanya oke. Saat itu
      pesen ayam bakar, jukut goreng, tempe sama pepes tahu. Ini semuanya enak
      (menurut pendapat pribadi).
  - text: >-
      babat:Kemaren kebetulan makan babat sama nyobain cumi, buat tekstur
      babatnya itu engga alot sama sekali dan tidak amis, sedangkan buat cumi
      utuh lumayan gede juga tekstur kenyel kenyelnya dapet dan mateng juga
      sampe ke dalem. 
pipeline_tag: text-classification
inference: false
model-index:
  - name: SetFit Aspect Model
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.80625
            name: Accuracy

SetFit Aspect Model

This is a SetFit model that can be used for Aspect Based Sentiment Analysis (ABSA). A LogisticRegression instance is used for classification. In particular, this model is in charge of filtering aspect span candidates.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

This model was trained within the context of a larger system for ABSA, which looks like so:

  1. Use a spaCy model to select possible aspect span candidates.
  2. Use this SetFit model to filter these possible aspect span candidates.
  3. Use a SetFit model to classify the filtered aspect span candidates.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
no aspect
  • 'ambel leuncanya:ambel leuncanya enak terus pedesss'
  • 'Warung Sunda:Warung Sunda murah meriah dan makanannya enak. Favorit selada air krispi dan ayam bakar'
  • 'makanannya:Warung Sunda murah meriah dan makanannya enak. Favorit selada air krispi dan ayam bakar'
aspect
  • 'ayam bakar:Warung Sunda murah meriah dan makanannya enak. Favorit selada air krispi dan ayam bakar'
  • 'Ayam bakar:Ayam bakar,sambel leunca sambel terasi merah enak banget 9/10, perkedel jagung 8/10 makan pakai sambel mantap. Makan berdua sekitar 77k'
  • 'sambel terasi merah:Ayam bakar,sambel leunca sambel terasi merah enak banget 9/10, perkedel jagung 8/10 makan pakai sambel mantap. Makan berdua sekitar 77k'

Evaluation

Metrics

Label Accuracy
all 0.8063

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import AbsaModel

# Download from the 🤗 Hub
model = AbsaModel.from_pretrained(
    "pahri/setfit-indo-resto-RM-ibu-imas-aspect",
    "pahri/setfit-indo-resto-RM-ibu-imas-polarity",
)
# Run inference
preds = model("The food was great, but the venue is just way too busy.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 4 37.7180 93
Label Training Sample Count
no aspect 371
aspect 51

Training Hyperparameters

  • batch_size: (6, 6)
  • num_epochs: (1, 16)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: True
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0000 1 0.4225 -
0.0021 50 0.2528 -
0.0043 100 0.3611 -
0.0064 150 0.2989 -
0.0085 200 0.2907 -
0.0107 250 0.1609 -
0.0128 300 0.3534 -
0.0149 350 0.1294 -
0.0171 400 0.2797 -
0.0192 450 0.3119 -
0.0213 500 0.004 -
0.0235 550 0.1057 -
0.0256 600 0.1049 -
0.0277 650 0.1601 -
0.0299 700 0.151 -
0.0320 750 0.1034 -
0.0341 800 0.2356 -
0.0363 850 0.1335 -
0.0384 900 0.0559 -
0.0405 950 0.0028 -
0.0427 1000 0.1307 -
0.0448 1050 0.0049 -
0.0469 1100 0.1348 -
0.0491 1150 0.0392 -
0.0512 1200 0.054 -
0.0533 1250 0.0016 -
0.0555 1300 0.0012 -
0.0576 1350 0.0414 -
0.0597 1400 0.1087 -
0.0618 1450 0.0464 -
0.0640 1500 0.0095 -
0.0661 1550 0.0011 -
0.0682 1600 0.0002 -
0.0704 1650 0.1047 -
0.0725 1700 0.001 -
0.0746 1750 0.0965 -
0.0768 1800 0.0002 -
0.0789 1850 0.1436 -
0.0810 1900 0.0011 -
0.0832 1950 0.001 -
0.0853 2000 0.1765 -
0.0874 2050 0.1401 -
0.0896 2100 0.0199 -
0.0917 2150 0.0 -
0.0938 2200 0.0023 -
0.0960 2250 0.0034 -
0.0981 2300 0.0001 -
0.1002 2350 0.0948 -
0.1024 2400 0.1634 -
0.1045 2450 0.0 -
0.1066 2500 0.0005 -
0.1088 2550 0.0695 -
0.1109 2600 0.0 -
0.1130 2650 0.0067 -
0.1152 2700 0.0025 -
0.1173 2750 0.0013 -
0.1194 2800 0.1426 -
0.1216 2850 0.0001 -
0.1237 2900 0.0 -
0.1258 2950 0.0 -
0.1280 3000 0.0001 -
0.1301 3050 0.0001 -
0.1322 3100 0.0122 -
0.1344 3150 0.0 -
0.1365 3200 0.0001 -
0.1386 3250 0.0041 -
0.1408 3300 0.2549 -
0.1429 3350 0.0062 -
0.1450 3400 0.0154 -
0.1472 3450 0.1776 -
0.1493 3500 0.0039 -
0.1514 3550 0.0183 -
0.1536 3600 0.0045 -
0.1557 3650 0.1108 -
0.1578 3700 0.0002 -
0.1600 3750 0.01 -
0.1621 3800 0.0002 -
0.1642 3850 0.0001 -
0.1664 3900 0.1612 -
0.1685 3950 0.0107 -
0.1706 4000 0.0548 -
0.1728 4050 0.0001 -
0.1749 4100 0.0162 -
0.1770 4150 0.1294 -
0.1792 4200 0.0 -
0.1813 4250 0.0032 -
0.1834 4300 0.0051 -
0.1855 4350 0.0 -
0.1877 4400 0.0151 -
0.1898 4450 0.0097 -
0.1919 4500 0.0002 -
0.1941 4550 0.0045 -
0.1962 4600 0.0001 -
0.1983 4650 0.0001 -
0.2005 4700 0.0227 -
0.2026 4750 0.0018 -
0.2047 4800 0.0 -
0.2069 4850 0.0001 -
0.2090 4900 0.0 -
0.2111 4950 0.0 -
0.2133 5000 0.0 -
0.2154 5050 0.0002 -
0.2175 5100 0.0002 -
0.2197 5150 0.0038 -
0.2218 5200 0.0 -
0.2239 5250 0.0 -
0.2261 5300 0.0 -
0.2282 5350 0.0028 -
0.2303 5400 0.0 -
0.2325 5450 0.1146 -
0.2346 5500 0.0 -
0.2367 5550 0.0073 -
0.2389 5600 0.0467 -
0.2410 5650 0.0092 -
0.2431 5700 0.0196 -
0.2453 5750 0.0002 -
0.2474 5800 0.0043 -
0.2495 5850 0.0378 -
0.2517 5900 0.0049 -
0.2538 5950 0.0054 -
0.2559 6000 0.1757 -
0.2581 6050 0.0 -
0.2602 6100 0.0001 -
0.2623 6150 0.1327 -
0.2645 6200 0.0 -
0.2666 6250 0.0 -
0.2687 6300 0.0 -
0.2709 6350 0.0134 -
0.2730 6400 0.0001 -
0.2751 6450 0.0112 -
0.2773 6500 0.0864 -
0.2794 6550 0.0 -
0.2815 6600 0.0094 -
0.2837 6650 0.1358 -
0.2858 6700 0.0155 -
0.2879 6750 0.0025 -
0.2901 6800 0.0002 -
0.2922 6850 0.0001 -
0.2943 6900 0.2809 -
0.2965 6950 0.0 -
0.2986 7000 0.0242 -
0.3007 7050 0.0015 -
0.3028 7100 0.0 -
0.3050 7150 0.1064 -
0.3071 7200 0.1636 -
0.3092 7250 0.267 -
0.3114 7300 0.1656 -
0.3135 7350 0.0943 -
0.3156 7400 0.189 -
0.3178 7450 0.0055 -
0.3199 7500 0.1286 -
0.3220 7550 0.1062 -
0.3242 7600 0.1275 -
0.3263 7650 0.0101 -
0.3284 7700 0.0162 -
0.3306 7750 0.0001 -
0.3327 7800 0.0001 -
0.3348 7850 0.0003 -
0.3370 7900 0.0 -
0.3391 7950 0.135 -
0.3412 8000 0.0 -
0.3434 8050 0.0125 -
0.3455 8100 0.0004 -
0.3476 8150 0.0 -
0.3498 8200 0.2229 -
0.3519 8250 0.0 -
0.3540 8300 0.0051 -
0.3562 8350 0.0 -
0.3583 8400 0.0001 -
0.3604 8450 0.0 -
0.3626 8500 0.1261 -
0.3647 8550 0.0054 -
0.3668 8600 0.1636 -
0.3690 8650 0.0036 -
0.3711 8700 0.0 -
0.3732 8750 0.0027 -
0.3754 8800 0.0 -
0.3775 8850 0.1422 -
0.3796 8900 0.1314 -
0.3818 8950 0.003 -
0.3839 9000 0.0 -
0.3860 9050 0.0092 -
0.3882 9100 0.0129 -
0.3903 9150 0.0 -
0.3924 9200 0.0 -
0.3946 9250 0.1659 -
0.3967 9300 0.0 -
0.3988 9350 0.0 -
0.4010 9400 0.0085 -
0.4031 9450 0.0 -
0.4052 9500 0.0 -
0.4074 9550 0.0 -
0.4095 9600 0.0112 -
0.4116 9650 0.0 -
0.4138 9700 0.0154 -
0.4159 9750 0.0011 -
0.4180 9800 0.0077 -
0.4202 9850 0.0064 -
0.4223 9900 0.0 -
0.4244 9950 0.0 -
0.4265 10000 0.0121 -
0.4287 10050 0.0 -
0.4308 10100 0.0 -
0.4329 10150 0.0076 -
0.4351 10200 0.0039 -
0.4372 10250 0.2153 -
0.4393 10300 0.0 -
0.4415 10350 0.1218 -
0.4436 10400 0.0077 -
0.4457 10450 0.1311 -
0.4479 10500 0.0 -
0.4500 10550 0.0 -
0.4521 10600 0.0 -
0.4543 10650 0.0041 -
0.4564 10700 0.0073 -
0.4585 10750 0.0051 -
0.4607 10800 0.0 -
0.4628 10850 0.0 -
0.4649 10900 0.0 -
0.4671 10950 0.0001 -
0.4692 11000 0.0 -
0.4713 11050 0.1696 -
0.4735 11100 0.0 -
0.4756 11150 0.1243 -
0.4777 11200 0.0 -
0.4799 11250 0.0 -
0.4820 11300 0.0003 -
0.4841 11350 0.0707 -
0.4863 11400 0.166 -
0.4884 11450 0.4964 -
0.4905 11500 0.0023 -
0.4927 11550 0.0 -
0.4948 11600 0.0 -
0.4969 11650 0.173 -
0.4991 11700 0.0 -
0.5012 11750 0.0004 -
0.5033 11800 0.0 -
0.5055 11850 0.125 -
0.5076 11900 0.0042 -
0.5097 11950 0.012 -
0.5119 12000 0.0046 -
0.5140 12050 0.0001 -
0.5161 12100 0.0062 -
0.5183 12150 0.0 -
0.5204 12200 0.017 -
0.5225 12250 0.2668 -
0.5247 12300 0.0986 -
0.5268 12350 0.0071 -
0.5289 12400 0.0055 -
0.5311 12450 0.006 -
0.5332 12500 0.0057 -
0.5353 12550 0.0044 -
0.5375 12600 0.0039 -
0.5396 12650 0.1685 -
0.5417 12700 0.125 -
0.5438 12750 0.0026 -
0.5460 12800 0.0 -
0.5481 12850 0.0 -
0.5502 12900 0.1024 -
0.5524 12950 0.0 -
0.5545 13000 0.0 -
0.5566 13050 0.0083 -
0.5588 13100 0.0 -
0.5609 13150 0.0001 -
0.5630 13200 0.0 -
0.5652 13250 0.095 -
0.5673 13300 0.0001 -
0.5694 13350 0.0026 -
0.5716 13400 0.0 -
0.5737 13450 0.0041 -
0.5758 13500 0.1654 -
0.5780 13550 0.0003 -
0.5801 13600 0.0056 -
0.5822 13650 0.0 -
0.5844 13700 0.1012 -
0.5865 13750 0.0 -
0.5886 13800 0.0001 -
0.5908 13850 0.0042 -
0.5929 13900 0.0122 -
0.5950 13950 0.1047 -
0.5972 14000 0.0 -
0.5993 14050 0.0121 -
0.6014 14100 0.0 -
0.6036 14150 0.0 -
0.6057 14200 0.0 -
0.6078 14250 0.0105 -
0.6100 14300 0.0 -
0.6121 14350 0.011 -
0.6142 14400 0.0329 -
0.6164 14450 0.0942 -
0.6185 14500 0.0173 -
0.6206 14550 0.0 -
0.6228 14600 0.1032 -
0.6249 14650 0.016 -
0.6270 14700 0.0079 -
0.6292 14750 0.0 -
0.6313 14800 0.1088 -
0.6334 14850 0.0091 -
0.6356 14900 0.0039 -
0.6377 14950 0.0 -
0.6398 15000 0.0 -
0.6420 15050 0.0 -
0.6441 15100 0.1654 -
0.6462 15150 0.0 -
0.6484 15200 0.0002 -
0.6505 15250 0.0 -
0.6526 15300 0.1745 -
0.6548 15350 0.0 -
0.6569 15400 0.156 -
0.6590 15450 0.0 -
0.6611 15500 0.0 -
0.6633 15550 0.1755 -
0.6654 15600 0.008 -
0.6675 15650 0.0 -
0.6697 15700 0.0 -
0.6718 15750 0.0041 -
0.6739 15800 0.0037 -
0.6761 15850 0.0 -
0.6782 15900 0.0 -
0.6803 15950 0.0092 -
0.6825 16000 0.0071 -
0.6846 16050 0.0053 -
0.6867 16100 0.0 -
0.6889 16150 0.004 -
0.6910 16200 0.0036 -
0.6931 16250 0.0 -
0.6953 16300 0.0 -
0.6974 16350 0.184 -
0.6995 16400 0.0 -
0.7017 16450 0.0133 -
0.7038 16500 0.0 -
0.7059 16550 0.174 -
0.7081 16600 0.0 -
0.7102 16650 0.0233 -
0.7123 16700 0.0117 -
0.7145 16750 0.0272 -
0.7166 16800 0.0095 -
0.7187 16850 0.0 -
0.7209 16900 0.1656 -
0.7230 16950 0.0055 -
0.7251 17000 0.0 -
0.7273 17050 0.1716 -
0.7294 17100 0.0 -
0.7315 17150 0.0 -
0.7337 17200 0.1035 -
0.7358 17250 0.0694 -
0.7379 17300 0.1733 -
0.7401 17350 0.0092 -
0.7422 17400 0.1656 -
0.7443 17450 0.0 -
0.7465 17500 0.1655 -
0.7486 17550 0.0059 -
0.7507 17600 0.1116 -
0.7529 17650 0.0 -
0.7550 17700 0.0068 -
0.7571 17750 0.0053 -
0.7593 17800 0.0 -
0.7614 17850 0.0062 -
0.7635 17900 0.0104 -
0.7657 17950 0.1727 -
0.7678 18000 0.0 -
0.7699 18050 0.0 -
0.7721 18100 0.0 -
0.7742 18150 0.0714 -
0.7763 18200 0.0 -
0.7785 18250 0.0 -
0.7806 18300 0.0002 -
0.7827 18350 0.0 -
0.7848 18400 0.0 -
0.7870 18450 0.0996 -
0.7891 18500 0.0 -
0.7912 18550 0.0 -
0.7934 18600 0.0139 -
0.7955 18650 0.0 -
0.7976 18700 0.1701 -
0.7998 18750 0.0 -
0.8019 18800 0.0001 -
0.8040 18850 0.0 -
0.8062 18900 0.0 -
0.8083 18950 0.0 -
0.8104 19000 0.0 -
0.8126 19050 0.0 -
0.8147 19100 0.1093 -
0.8168 19150 0.0 -
0.8190 19200 0.0 -
0.8211 19250 0.0075 -
0.8232 19300 0.1079 -
0.8254 19350 0.0112 -
0.8275 19400 0.1655 -
0.8296 19450 0.0152 -
0.8318 19500 0.1152 -
0.8339 19550 0.0 -
0.8360 19600 0.0 -
0.8382 19650 0.0079 -
0.8403 19700 0.0 -
0.8424 19750 0.0 -
0.8446 19800 0.0 -
0.8467 19850 0.0 -
0.8488 19900 0.1161 -
0.8510 19950 0.0057 -
0.8531 20000 0.0 -
0.8552 20050 0.0046 -
0.8574 20100 0.0 -
0.8595 20150 0.0068 -
0.8616 20200 0.0 -
0.8638 20250 0.0 -
0.8659 20300 0.0 -
0.8680 20350 0.0 -
0.8702 20400 0.0141 -
0.8723 20450 0.0001 -
0.8744 20500 0.0 -
0.8766 20550 0.0 -
0.8787 20600 0.0171 -
0.8808 20650 0.0 -
0.8830 20700 0.0 -
0.8851 20750 0.0077 -
0.8872 20800 0.0 -
0.8894 20850 0.0 -
0.8915 20900 0.0 -
0.8936 20950 0.0 -
0.8958 21000 0.0 -
0.8979 21050 0.0 -
0.9000 21100 0.0 -
0.9021 21150 0.0 -
0.9043 21200 0.0 -
0.9064 21250 0.1048 -
0.9085 21300 0.006 -
0.9107 21350 0.0 -
0.9128 21400 0.0 -
0.9149 21450 0.005 -
0.9171 21500 0.0 -
0.9192 21550 0.0325 -
0.9213 21600 0.0136 -
0.9235 21650 0.0 -
0.9256 21700 0.0062 -
0.9277 21750 0.1656 -
0.9299 21800 0.1648 -
0.9320 21850 0.0 -
0.9341 21900 0.0 -
0.9363 21950 0.0 -
0.9384 22000 0.2844 -
0.9405 22050 0.0 -
0.9427 22100 0.0 -
0.9448 22150 0.0 -
0.9469 22200 0.0 -
0.9491 22250 0.0 -
0.9512 22300 0.2096 -
0.9533 22350 0.0073 -
0.9555 22400 0.006 -
0.9576 22450 0.0 -
0.9597 22500 0.0079 -
0.9619 22550 0.0071 -
0.9640 22600 0.0 -
0.9661 22650 0.006 -
0.9683 22700 0.1048 -
0.9704 22750 0.007 -
0.9725 22800 0.0 -
0.9747 22850 0.0 -
0.9768 22900 0.007 -
0.9789 22950 0.0 -
0.9811 23000 0.1049 -
0.9832 23050 0.0069 -
0.9853 23100 0.0 -
0.9875 23150 0.0 -
0.9896 23200 0.0 -
0.9917 23250 0.0 -
0.9939 23300 0.007 -
0.9960 23350 0.0147 -
0.9981 23400 0.0 -

Framework Versions

  • Python: 3.10.13
  • SetFit: 1.0.3
  • Sentence Transformers: 2.7.0
  • spaCy: 3.7.4
  • Transformers: 4.36.2
  • PyTorch: 2.1.2
  • Datasets: 2.18.0
  • Tokenizers: 0.15.2

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}