metadata
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
metrics:
- Precision_micro
- Precision_weighted
- Precision_samples
- Recall_micro
- Recall_weighted
- Recall_samples
- F1-Score
- accuracy
widget:
- text: >-
Violence from intimate partners and male family members can escalate
during emergencies. This tends to increase as the crisis worsens, and men
have lost their jobs and status – particularly in communities with
traditional gender roles, and where family violence is normalised
- text: >-
Expand livelihood protection policies that assist vulnerable, low-income
individuals to recover from damages associated with extreme weather
events; provide support and protection for internally displaced persons,
persons displaced across borders and host communities;. By 2026, draw up
disaster recovery plans for all 22 municipalities with resource
inventories, first response measures and actions (including on logistics)
concerning humanitarian post-disaster needs.
- text: >-
recurrent droughts, (decrease in amount of rainfall from 550 to 400mm in
the highlands), changes in seasonality that had resulted frequent crop
failure, massive death of livestock, genetic erosion, extinction of
endemic species, degradation of habitats and disequilibria in the
ecosystem structure and function. The impact of climate change is
manifested in recurrent droughts, desertification, sea level rise and
increase in sea water temperature, depletion of ground water, widespread
land degradation, and emergence of climate sensitive diseases.
- text: >-
They live in geographical regions and ecosystems that are the most
vulnerable to climate change. These include polar regions, humid tropical
forests, high mountains, small islands, coastal regions, and arid and
semi-arid lands, among others. The impacts of climate change in such
regions have strong implications for the ecosystem-based livelihoods on
which many indigenous peoples depend. Moreover, in some regions such as
the Pacific, the very existence of many indigenous territories is under
threat from rising sea levels that not only pose a grave threat to
indigenous peoples’ livelihoods but also to their cultures and ways of
life.
- text: >-
Overcoming Poverty. Colombia, as a developing country, faces major
socioeconomic challenges. According to the official figures of DANE, by
2014, the percentage of people in multidimensional poverty situation was
21.9% (this figure rises to 44.1% if we take into account only the rural
population). For the same year, 28.5% of the population was found in a
situation of monetary poverty (41.4% of the population in the case of the
villages and rural centers scattered).
pipeline_tag: text-classification
inference: false
base_model: sentence-transformers/all-mpnet-base-v2
model-index:
- name: SetFit with sentence-transformers/all-mpnet-base-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: Precision_micro
value: 0.7972027972027972
name: Precision_Micro
- type: Precision_weighted
value: 0.8053038510784989
name: Precision_Weighted
- type: Precision_samples
value: 0.7972027972027972
name: Precision_Samples
- type: Recall_micro
value: 0.7972027972027972
name: Recall_Micro
- type: Recall_weighted
value: 0.7972027972027972
name: Recall_Weighted
- type: Recall_samples
value: 0.7972027972027972
name: Recall_Samples
- type: F1-Score
value: 0.7972027972027972
name: F1-Score
- type: accuracy
value: 0.7972027972027972
name: Accuracy
SetFit with sentence-transformers/all-mpnet-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
- Classification head: a OneVsRestClassifier instance
- Maximum Sequence Length: 384 tokens
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Evaluation
Metrics
| Label | Precision_Micro | Precision_Weighted | Precision_Samples | Recall_Micro | Recall_Weighted | Recall_Samples | F1-Score | Accuracy |
|---|---|---|---|---|---|---|---|---|
| all | 0.7972 | 0.8053 | 0.7972 | 0.7972 | 0.7972 | 0.7972 | 0.7972 | 0.7972 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("leavoigt/vulnerability_target")
# Run inference
preds = model("Violence from intimate partners and male family members can escalate during emergencies. This tends to increase as the crisis worsens, and men have lost their jobs and status – particularly in communities with traditional gender roles, and where family violence is normalised")
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Word count | 15 | 71.9518 | 238 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 20
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
| Epoch | Step | Training Loss | Validation Loss |
|---|---|---|---|
| 0.0012 | 1 | 0.2559 | - |
| 0.0602 | 50 | 0.2509 | - |
| 0.1205 | 100 | 0.2595 | - |
| 0.1807 | 150 | 0.0868 | - |
| 0.2410 | 200 | 0.0302 | - |
| 0.3012 | 250 | 0.0024 | - |
| 0.3614 | 300 | 0.0225 | - |
| 0.4217 | 350 | 0.0007 | - |
| 0.4819 | 400 | 0.0004 | - |
| 0.5422 | 450 | 0.0003 | - |
| 0.6024 | 500 | 0.0002 | - |
| 0.6627 | 550 | 0.0005 | - |
| 0.7229 | 600 | 0.0319 | - |
| 0.7831 | 650 | 0.0001 | - |
| 0.8434 | 700 | 0.0104 | - |
| 0.9036 | 750 | 0.0003 | - |
| 0.9639 | 800 | 0.0009 | - |
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.1
- Sentence Transformers: 2.2.2
- Transformers: 4.25.1
- PyTorch: 2.1.0+cu121
- Datasets: 2.16.1
- Tokenizers: 0.13.3
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}