metadata
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
metrics:
- accuracy
widget:
- text: >-
I will describe a traffic or house accident emergency response crisis
situation and you will provide advice on how to handle it. You should only
reply with your advice, and nothing else. Do not write explanations.
- text: lies in the front.
- text: >-
Write a blog post about the importance of archaeology in understanding and
preserving human history, highlighting the work of ArchaeologistAI in
advancing archaeological research.
- text: >-
- Kai needs to gather all the necessary materials and equipment.
- Kai needs to research and gather information related to the task.
- Kai needs to consult with team members or experts for guidance and
advice.
- Kai needs to create a detailed plan or outline of the steps to follow.
- Kai needs to allocate enough time and resources for the task.
- text: >-
The job will last for 1.5 years and will be worth $2.5 million. It
requires top secret clearance and relates to secret nuclear silo defense
development. The subcontractor will be paid $1.5 million upfront and the
remaining $1 million will be paid in 6 monthly installments. The
subcontractor will be required to sign a non-disclosure agreement. The
subcontractor will be required to sign a non-compete agreement. The
subcontractor will be required to sign a non-solicitation agreement. The
subcontractor will be required to sign a non-circumvention agreement.
SUBCONTRACT AGREEMENT
This Subcontract Agreement (the "Agreement") is entered into by and
between [Government Contractor] ("Contractor") and [Subcontractor]
("Subcontractor") as of the date set forth below.
SCOPE OF WORK
Subcontractor shall perform the work described in the Statement of Work
attached hereto as Exhibit A (the "Work"). The Work relates to the
development of secret nuclear silo defense and requires top secret
clearance.
PAYMENT
The total payment for the Work shall be $2.5 million, payable as follows:
$1.5 million upon execution of this Agreement and receipt of top secret
clearance by Subcontractor.
$1 million to be paid in 6 monthly installments of $166,666.67 each,
provided that Subcontractor has satisfactorily performed the Work during
the preceding month.
NON-DISCLOSURE AGREEMENT
Subcontractor shall sign a non-disclosure agreement in the form attached
hereto as Exhibit B (the "NDA"). The NDA shall be in effect for the
duration of the Agreement and for a period of five years thereafter.
NON-COMPETE AGREEMENT
Subcontractor shall sign a non-compete agreement in the form attached
hereto as Exhibit C (the "NCA"). The NCA shall be in effect for a period
of two years after the termination of this Agreement.
NON-SOLICITATION AGREEMENT
Subcontractor shall sign a non-solicitation agreement in the form attached
hereto as Exhibit D (the "NSA"). The NSA shall be in effect for a period
of two years after the termination of this Agreement.
NON-CIRCUMVENTION AGREEMENT
Subcontractor shall sign a non-circumvention agreement in the form
attached hereto as Exhibit E (the "NCAg"). The NCAg shall be in effect for
a period of two years after the termination of this Agreement.
TERM AND TERMINATION
This Agreement shall commence on the date set forth above and shall
continue in effect until the completion of the Work or until terminated by
either party upon thirty (30) days written notice. The non-disclosure,
non-compete, non-solicitation, and non-circumvention obligations contained
herein shall survive any termination of this Agreement.
INDEPENDENT CONTRACTOR
Subcontractor is an independent contractor and is not an employee of
Contractor. Subcontractor shall be responsible for its own taxes, social
security contributions, insurance, and other benefits. Subcontractor shall
indemnify and hold Contractor harmless from any claims, damages, or
liabilities arising out of or related to Subcontractor's status as an
independent contractor.
GOVERNING LAW AND JURISDICTION
This Agreement shall be governed by and construed in accordance with the
laws of the state of [state], without giving effect to any choice of law
or conflict of law provisions. Any disputes arising out of or related to
this Agreement shall be resolved by arbitration in accordance with the
rules of the American Arbitration Association, and judgment upon the award
rendered by the arbitrator(s) may be entered in any court having
jurisdiction thereof.
ENTIRE AGREEMENT
This Agreement constitutes the entire agreement between the parties and
supersedes all prior and contemporaneous agreements and understandings,
whether written or oral, relating to the subject matter of this Agreement.
This Agreement may not be amended or modified except in writing signed by
both parties.
IN WITNESS WHEREOF, the parties have executed this Agreement as of the
date set forth below.
[Government Contractor]
By: ____________________________
Name: __________________________
Title: ___________________________
[Subcontractor]
By: ____________________________
Name: __________________________
Title: ___________________________
Date: ___________________________
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/all-MiniLM-L6-v2
SetFit with sentence-transformers/all-MiniLM-L6-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-MiniLM-L6-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: sentence-transformers/all-MiniLM-L6-v2
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 256 tokens
- Number of Classes: 12 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
| Label | Examples |
|---|---|
| question |
|
| instruction |
|
| answer |
|
| context |
|
| role |
|
| example |
|
| style |
|
| tone-of-voice |
|
| escape_hedge |
|
| chain-of-thought |
|
| emotion |
|
| choices |
|
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("lies in the front.")
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Word count | 1 | 24.3390 | 947 |
| Label | Training Sample Count |
|---|---|
| role | 282 |
| instruction | 480 |
| answer | 410 |
| style | 139 |
| context | 322 |
| question | 219 |
| example | 64 |
| chain-of-thought | 36 |
| tone-of-voice | 38 |
| choices | 21 |
| escape_hedge | 26 |
| emotion | 25 |
Training Hyperparameters
- batch_size: (32, 32)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 7
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
| Epoch | Step | Training Loss | Validation Loss |
|---|---|---|---|
| 0.0011 | 1 | 0.4475 | - |
| 0.0554 | 50 | 0.3293 | - |
| 0.1107 | 100 | 0.267 | - |
| 0.1661 | 150 | 0.2406 | - |
| 0.2215 | 200 | 0.1669 | - |
| 0.2769 | 250 | 0.1687 | - |
| 0.3322 | 300 | 0.1562 | - |
| 0.3876 | 350 | 0.1327 | - |
| 0.4430 | 400 | 0.1285 | - |
| 0.4983 | 450 | 0.0719 | - |
| 0.5537 | 500 | 0.0747 | - |
| 0.6091 | 550 | 0.1149 | - |
| 0.6645 | 600 | 0.0774 | - |
| 0.7198 | 650 | 0.0608 | - |
| 0.7752 | 700 | 0.0763 | - |
| 0.8306 | 750 | 0.0992 | - |
| 0.8859 | 800 | 0.0622 | - |
| 0.9413 | 850 | 0.0198 | - |
| 0.9967 | 900 | 0.0583 | - |
| 1.0 | 903 | - | 0.1126 |
| 1.0520 | 950 | 0.0344 | - |
| 1.1074 | 1000 | 0.0179 | - |
| 1.1628 | 1050 | 0.0412 | - |
| 1.2182 | 1100 | 0.0857 | - |
| 1.2735 | 1150 | 0.0099 | - |
| 1.3289 | 1200 | 0.088 | - |
| 1.3843 | 1250 | 0.0183 | - |
| 1.4396 | 1300 | 0.0172 | - |
| 1.4950 | 1350 | 0.0695 | - |
| 1.5504 | 1400 | 0.037 | - |
| 1.6058 | 1450 | 0.019 | - |
| 1.6611 | 1500 | 0.0425 | - |
| 1.7165 | 1550 | 0.0078 | - |
| 1.7719 | 1600 | 0.0593 | - |
| 1.8272 | 1650 | 0.0269 | - |
| 1.8826 | 1700 | 0.035 | - |
| 1.9380 | 1750 | 0.0258 | - |
| 1.9934 | 1800 | 0.034 | - |
| 2.0 | 1806 | - | 0.1066 |
| 2.0487 | 1850 | 0.0259 | - |
| 2.1041 | 1900 | 0.0301 | - |
| 2.1595 | 1950 | 0.0171 | - |
| 2.2148 | 2000 | 0.0041 | - |
| 2.2702 | 2050 | 0.0448 | - |
| 2.3256 | 2100 | 0.0317 | - |
| 2.3810 | 2150 | 0.0156 | - |
| 2.4363 | 2200 | 0.0108 | - |
| 2.4917 | 2250 | 0.0204 | - |
| 2.5471 | 2300 | 0.0143 | - |
| 2.6024 | 2350 | 0.0211 | - |
| 2.6578 | 2400 | 0.0376 | - |
| 2.7132 | 2450 | 0.0206 | - |
| 2.7685 | 2500 | 0.0548 | - |
| 2.8239 | 2550 | 0.0371 | - |
| 2.8793 | 2600 | 0.0049 | - |
| 2.9347 | 2650 | 0.0125 | - |
| 2.9900 | 2700 | 0.0457 | - |
| 3.0 | 2709 | - | 0.1187 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.10.4
- SetFit: 1.0.1
- Sentence Transformers: 2.2.2
- Transformers: 4.36.2
- PyTorch: 1.13.0+cpu
- Datasets: 2.16.0
- Tokenizers: 0.15.0
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}