Edit model card

SetFit with ppsingh/SECTOR-multilabel-mpnet_w

This is a SetFit model that can be used for Text Classification. This SetFit model uses ppsingh/SECTOR-multilabel-mpnet_w as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("ppsingh/iki_sector_setfit")
# Run inference
preds = model("In the shipping and aviation sectors, emission reduction efforts will be focused on distributing eco-friendly ships and enhancing the operational efficiency of aircraft. Agriculture, livestock farming and fisheries: The Republic Korea is introducing various options to accelerate low-carbon farming, for instance, improving irrigation techniques in rice paddies and adopting low-input systems for nitrogen fertilizers.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 35 76.164 170
  • Training Dataset: 250

    Class Positive Count of Class
    Economy-wide 88
    Energy 63
    Other Sector 64
    Transport 139
  • Validation Dataset: 42

    Class Positive Count of Class
    Economy-wide 15
    Energy 11
    Other Sector 11
    Transport 24

Training Hyperparameters

  • batch_size: (16, 2)
  • num_epochs: (1, 10)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.01
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0005 1 0.2029 -
0.0993 200 0.0111 0.1124
0.1985 400 0.0063 0.111
0.2978 600 0.0183 0.1214
0.3970 800 0.0197 0.1248
0.4963 1000 0.0387 0.1339
0.5955 1200 0.0026 0.1181
0.6948 1400 0.0378 0.1208
0.7940 1600 0.0285 0.1267
0.8933 1800 0.0129 0.1254
0.9926 2000 0.0341 0.1271

Classifier Training Results

Epoch Training F1-micro Training F1-Samples Training F1-weighted Validation F1-micro Validation F1-samples Validation F1-weighted
0 0.954 0.972 0.945 0.824 0.819 0.813
1 0.994 0.996 0.994 0.850 0.832 0.852
2 0.981 0.989 0.979 0.850 0.843 0.852
3 0.995 0.997 0.995 0.852 0.843 0.858
4 0.994 0.996 0.994 0.852 0.843 0.858
5 0.995 0.997 0.995 0.859 0.848 0.863
label precision recall f1-score support
Economy-wide 0.857 0.800 0.827 15.0
Energy 1.00 0.818 0.900 11.0
Other Sector 0.615 0.727 0.667 11.0
Transport 0.958 0.958 0.958 24.0
  • Micro Avg: Precision = 0.866, Recall = 0.852, F1 = 0.859504
  • Samples Avg: Precision = 0.869, Recall = 0.861, F1 = 0.848

Environmental Impact

Carbon emissions were measured using CodeCarbon.

  • Carbon Emitted: 0.026 kg of CO2
  • Hours Used: 0.622 hours

Training Hardware

  • On Cloud: No
  • GPU Model: 1 x Tesla T4
  • CPU Model: Intel(R) Xeon(R) CPU @ 2.00GHz
  • RAM Size: 12.67 GB

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 2.3.1
  • Transformers: 4.35.2
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.3.0
  • Tokenizers: 0.15.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
1
Safetensors
Model size
109M params
Tensor type
F32
·
Inference API (serverless) has been turned off for this model.
Invalid base_model specified in model card metadata. Needs to be a model id from hf.co/models.