Hugging Face's logo

babysharkdododo
/
setfit-all-minilm-l6-v2-malay_en_cn_sentiment_analysis

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results
Model card Files Community
setfit-all-minilm-l6-v2-malay_en_cn_sentiment_analysis / README.md
babysharkdododo's picture
babysharkdododo
Add SetFit model
c1a73df verified
preview code
|
raw
history blame
19 kB
metadata
library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - f1
widget:
  - text: >-
      Kerajaan bukannya memandai-mandai buat itu ini, sebaliknya yang
      dilaksanakan adalah bagi penuhi permintaan atau cadangan diterima daripada
      peringkat bawahan sendiri
  - text: >-
      mahathir mohamad demi kelangsungan karier politiknya lebih-lebih lagi
      bekas perdana menteri itu masih lagi mempunyai pengikut yang taksub
  - text: >-
      @AINAMIR96 Bukan..kalau letak mmg lah melecur..ambik towel kecik..iron bg
      panas..lepas tu tuam lah kat perut towel https://t.co/pAw4o5vr5I
  - text: >-
      Kebanyakan orang Cina yang kamu temui di sini akan beritahu kamu,
      kebimbangan mereka tentang isu yang melanda negara.
  - text: >-
      WTB | WHAT TO BUY TAEIL BATU AKIK ( FIRE TRUCK ) dm aku yaa chagi kali aja
      ada yg mau jual taeilnya ini bener https://t.co/el2UKgB3j4
pipeline_tag: text-classification
inference: true
model-index:
  - name: SetFit
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: f1
            value: 0.608
            name: F1

SetFit

This is a SetFit model that can be used for Text Classification. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

  • Model Type: SetFit
  • Classification head: a LogisticRegression instance
  • Maximum Sequence Length: 256 tokens
  • Number of Classes: 3 classes

Model Sources

Model Labels

Label Examples
negative
  • 'Biasanya cuma janji saja tapi tak pernah buat pun'
  • 'Aku faham kalau diorang buat macam ni sebab kalau aku jadi doktor pun aku penat la kimak. Bodoh punya kerajaan !'
  • 'Jika parlimen tidak dipanggil dalam masa 14 hari maka UMNO akan keluar menjadi pembangkang iaitu lebih kurang 4 Jul https://t.co/AvJSf1F8ux'
positive
  • 'Kek telapuk kuda, kek lembut rasa premium! Tgh kumpoi order ni utk warga Sungai Petani. Tak terlioq ka tengok? Mai https://t.co/wgcEADvxQK'
  • 'Kalau projek ini berjalan, peneroka Felda tak payah masuk kebun lagi, harga sawit baru RM500 1 tan, murah.'
  • 'Justeru, kita berharap lebih banyak pusat rawatan seperti ini diwujudkan untuk membantu.'
neutral
  • '08.05 WIB #Jalan_Layang_MBZ Cikunir - Tambun - Cikarang - Karawang LANCAR. ; Karawang - Cikarang - Tambun - Cikunir LANCAR.'
  • '5) Menilai Kualiti Kandungan. Selepas dah faham apa yang penulis cuba sampaikan, kita boleh nilai sama ada ia bena https://t.co/LnkjtBc3Nm'
  • '@syafirazbd_ Siapp'

Evaluation

Metrics

Label F1
all 0.608

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("babysharkdododo/setfit-all-minilm-l6-v2-malay_en_cn_sentiment_analysis")
# Run inference
preds = model("Kebanyakan orang Cina yang kamu temui di sini akan beritahu kamu, kebimbangan mereka tentang isu yang melanda negara.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 16.598 58
Label Training Sample Count
positive 235
neutral 77
negative 188

Training Hyperparameters

  • batch_size: (128, 128)
  • num_epochs: (10, 10)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0001 1 0.189 -
0.0052 50 0.3053 -
0.0104 100 0.2779 -
0.0156 150 0.4526 -
0.0208 200 0.3073 -
0.0261 250 0.4156 -
0.0313 300 0.3912 -
0.0365 350 0.2259 -
0.0417 400 0.2445 -
0.0004 1 0.3311 -
0.0208 50 0.3569 -
0.0417 100 0.2946 -
0.0625 150 0.3397 -
0.0008 1 0.3177 -
0.0417 50 0.2764 -
0.0833 100 0.226 -
0.125 150 0.2649 -
0.1667 200 0.282 -
0.2083 250 0.2438 -
0.25 300 0.2425 -
0.2917 350 0.2555 -
0.3333 400 0.2266 -
0.375 450 0.14 -
0.4167 500 0.1446 -
0.4583 550 0.152 -
0.5 600 0.184 -
0.5417 650 0.095 -
0.5833 700 0.1358 -
0.625 750 0.0859 -
0.6667 800 0.0756 -
0.7083 850 0.0622 -
0.75 900 0.0719 -
0.7917 950 0.0681 -
0.8333 1000 0.0684 -
0.875 1050 0.0356 -
0.9167 1100 0.0233 -
0.9583 1150 0.0126 -
1.0 1200 0.0022 0.3748
1.0417 1250 0.0095 -
1.0833 1300 0.0095 -
1.125 1350 0.0376 -
1.1667 1400 0.0075 -
1.2083 1450 0.0075 -
1.25 1500 0.0142 -
1.2917 1550 0.0113 -
1.3333 1600 0.0022 -
1.375 1650 0.0006 -
1.4167 1700 0.0005 -
1.4583 1750 0.0005 -
1.5 1800 0.0003 -
1.5417 1850 0.0021 -
1.5833 1900 0.0004 -
1.625 1950 0.0006 -
1.6667 2000 0.001 -
1.7083 2050 0.0002 -
1.75 2100 0.0002 -
1.7917 2150 0.0002 -
1.8333 2200 0.0002 -
1.875 2250 0.0059 -
1.9167 2300 0.0002 -
1.9583 2350 0.0005 -
2.0 2400 0.0001 0.3806
2.0417 2450 0.0001 -
2.0833 2500 0.0012 -
2.125 2550 0.0001 -
2.1667 2600 0.0002 -
2.2083 2650 0.0002 -
2.25 2700 0.0001 -
2.2917 2750 0.0011 -
2.3333 2800 0.0002 -
2.375 2850 0.0001 -
2.4167 2900 0.0003 -
2.4583 2950 0.0007 -
2.5 3000 0.0001 -
2.5417 3050 0.0001 -
2.5833 3100 0.0001 -
2.625 3150 0.0001 -
2.6667 3200 0.0001 -
2.7083 3250 0.0001 -
2.75 3300 0.0001 -
2.7917 3350 0.0002 -
2.8333 3400 0.0001 -
2.875 3450 0.0001 -
2.9167 3500 0.0001 -
2.9583 3550 0.0001 -
3.0 3600 0.0001 0.4004
3.0417 3650 0.0001 -
3.0833 3700 0.0001 -
3.125 3750 0.0001 -
3.1667 3800 0.0001 -
3.2083 3850 0.0002 -
3.25 3900 0.0001 -
3.2917 3950 0.0001 -
3.3333 4000 0.0005 -
3.375 4050 0.0001 -
3.4167 4100 0.0001 -
3.4583 4150 0.0001 -
3.5 4200 0.0004 -
3.5417 4250 0.0 -
3.5833 4300 0.0001 -
3.625 4350 0.0001 -
3.6667 4400 0.0001 -
3.7083 4450 0.0001 -
3.75 4500 0.0 -
3.7917 4550 0.0 -
3.8333 4600 0.0 -
3.875 4650 0.0001 -
3.9167 4700 0.0001 -
3.9583 4750 0.0001 -
4.0 4800 0.0 0.4004
4.0417 4850 0.0001 -
4.0833 4900 0.0003 -
4.125 4950 0.0 -
4.1667 5000 0.0001 -
4.2083 5050 0.0001 -
4.25 5100 0.0 -
4.2917 5150 0.0003 -
4.3333 5200 0.0001 -
4.375 5250 0.0 -
4.4167 5300 0.0 -
4.4583 5350 0.0002 -
4.5 5400 0.0 -
4.5417 5450 0.0001 -
4.5833 5500 0.0001 -
4.625 5550 0.0 -
4.6667 5600 0.0006 -
4.7083 5650 0.0 -
4.75 5700 0.0 -
4.7917 5750 0.0 -
4.8333 5800 0.0 -
4.875 5850 0.0 -
4.9167 5900 0.0 -
4.9583 5950 0.0 -
5.0 6000 0.0001 0.391
5.0417 6050 0.0 -
5.0833 6100 0.0001 -
5.125 6150 0.0 -
5.1667 6200 0.0 -
5.2083 6250 0.0 -
5.25 6300 0.0 -
5.2917 6350 0.0 -
5.3333 6400 0.0 -
5.375 6450 0.0 -
5.4167 6500 0.0 -
5.4583 6550 0.0 -
5.5 6600 0.0001 -
5.5417 6650 0.0 -
5.5833 6700 0.0 -
5.625 6750 0.0 -
5.6667 6800 0.0 -
5.7083 6850 0.0 -
5.75 6900 0.0001 -
5.7917 6950 0.0 -
5.8333 7000 0.0001 -
5.875 7050 0.0 -
5.9167 7100 0.0 -
5.9583 7150 0.0 -
6.0 7200 0.0001 0.4026
6.0417 7250 0.0 -
6.0833 7300 0.0 -
6.125 7350 0.0 -
6.1667 7400 0.0 -
6.2083 7450 0.0 -
6.25 7500 0.0 -
6.2917 7550 0.0 -
6.3333 7600 0.0 -
6.375 7650 0.0 -
6.4167 7700 0.0 -
6.4583 7750 0.0 -
6.5 7800 0.0 -
6.5417 7850 0.0 -
6.5833 7900 0.0 -
6.625 7950 0.0 -
6.6667 8000 0.0001 -
6.7083 8050 0.0005 -
6.75 8100 0.0063 -
6.7917 8150 0.0 -
6.8333 8200 0.0 -
6.875 8250 0.0 -
6.9167 8300 0.0 -
6.9583 8350 0.0 -
7.0 8400 0.0 0.4018
7.0417 8450 0.0 -
7.0833 8500 0.0 -
7.125 8550 0.0 -
7.1667 8600 0.0 -
7.2083 8650 0.0 -
7.25 8700 0.0 -
7.2917 8750 0.0 -
7.3333 8800 0.0 -
7.375 8850 0.0 -
7.4167 8900 0.0 -
7.4583 8950 0.0 -
7.5 9000 0.0 -
7.5417 9050 0.0 -
7.5833 9100 0.0 -
7.625 9150 0.0 -
7.6667 9200 0.0 -
7.7083 9250 0.0 -
7.75 9300 0.0 -
7.7917 9350 0.0 -
7.8333 9400 0.0 -
7.875 9450 0.0 -
7.9167 9500 0.0 -
7.9583 9550 0.0 -
8.0 9600 0.0 0.4001
8.0417 9650 0.0 -
8.0833 9700 0.0 -
8.125 9750 0.0 -

Framework Versions

  • Python: 3.10.14
  • SetFit: 1.0.3
  • Sentence Transformers: 3.0.1
  • Transformers: 4.39.0
  • PyTorch: 2.2.2
  • Datasets: 2.20.0
  • Tokenizers: 0.15.2

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}