Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,215 @@
+---
+library_name: setfit
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+base_model: sentence-transformers/all-roberta-large-v1
+metrics:
+- accuracy
+widget:
+- text: ' Anytime I’d feel uncomfortable in gatherings I’d be like “haha fuck this
+    I’m OUTTTTTAAAA HEEERRREEE” then go outside to recollect myself for a few moments'
+- text: ' But even then, I didn''t have any other problems outside school I still
+    had no friends at European school, I haven''t had any walks which I had constantly
+    with my friends back in Ukraine'
+- text: ' I do try to get out a lot because of this especially now that it''s nicer
+    outside I go for runs every day almost but this last week like something threw
+    me off'
+- text: ' I literally do not leave the house, I will sometimes go in the garden but
+    no further So I just ran outside and back to the car'
+- text: ' I hated even going outside and seeing other people, because I just knew
+    how awkward and weird they would think I am for the way I looked and the way I
+    walked I hated even going outside and seeing other people, because I just knew
+    how awkward and weird they would think I am for the way I looked and the way I
+    walked'
+pipeline_tag: text-classification
+inference: true
+model-index:
+- name: SetFit with sentence-transformers/all-roberta-large-v1
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.43333333333333335
+      name: Accuracy
+---
+
+# SetFit with sentence-transformers/all-roberta-large-v1
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-roberta-large-v1](https://huggingface.co/sentence-transformers/all-roberta-large-v1) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/all-roberta-large-v1](https://huggingface.co/sentence-transformers/all-roberta-large-v1)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 256 tokens
+- **Number of Classes:** 4 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+|:------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 3.0   | <ul><li>' However, one day after lunch I was walking to his classroom to get to his class Every time I go outside I instantly get nervous'</li><li>' Very generic title, but i was just wondering, does anyone feel really safe and happy when hiding? For example everytime i go outside for work i get stared at'</li><li>" Also, i'm living in my parent's home for 20 years and although i know very few neighbors i feel stalked outside, and it's like everyone knows who i am (the building complex is in a dead-end street and all the windows make me anxious) Today i was leaving school's parking lot when i gave way to 2 other students - one looked at me in complete disgust while passing by, the other laughed at me (i even considered if i made some mistake) So i parked a bit more inclined to the road side than to the sidewalk (these are small streets and people usually park in the sidewalk) and other car couldn't pass to the end of the street"</li></ul> |
+| 0.0   | <ul><li>' Good to know that some people are in the same camp'</li><li>' If anything you should be thinking about wearing sun screen so you retain your good skin as it becomes your ally as you age outside'</li><li>" I'm sure if the worlds ever did clash that your friends would understand (few people actually enjoy being at work) and, worst case scenario, your coworkers would be surprised at your outgoing nature while around friends"</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+| 1.0   | <ul><li>" Now I'm not saying to go to a party on the spot, just go out, shop, take a walk in the park, that kind of thing Now I'm not saying to go to a party on the spot, just go out, shop, take a walk in the park, that kind of thing"</li><li>" Dairy can help a lot too! And for the boost and help that exercise makes, Go for jogs at night assuming you live in a safe area! Thats what I do! Between 1am and 3am nobody is ever out In my months of walking at night I've never seen someone even look at me from their car"</li><li>" That's kind of the nature of my volunteer work, but you could volunteer with a food bank or boys  and girls club, which would involve more social interaction Just breaking that cycle by going for a short walk around the neighbourhood is a good idea"</li></ul>                                                                                                                                                                     |
+| 2.0   | <ul><li>' It could be 100 degrees outside and I refused to take the hoodie off'</li><li>' I hated even going outside and seeing other people, because I just knew how awkward and weird they would think I am for the way I looked and the way I walked I hated even going outside and seeing other people, because I just knew how awkward and weird they would think I am for the way I looked and the way I walked'</li><li>' I like art and nature but you can’t really talk about those for more than a few seconds'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 0.4333   |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("Omar-Nasr/setfitmodel")
+# Run inference
+preds = model(" I literally do not leave the house, I will sometimes go in the garden but no further So I just ran outside and back to the car")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median | Max |
+|:-------------|:----|:-------|:----|
+| Word count   | 7   | 33.15  | 121 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| 0.0   | 10                    |
+| 1.0   | 10                    |
+| 2.0   | 10                    |
+| 3.0   | 10                    |
+
+### Training Hyperparameters
+- batch_size: (16, 16)
+- num_epochs: (4, 4)
+- max_steps: -1
+- sampling_strategy: oversampling
+- body_learning_rate: (2e-05, 1e-05)
+- head_learning_rate: 0.01
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0133 | 1    | 0.2699        | -               |
+| 0.6667 | 50   | 0.0497        | -               |
+| 1.3333 | 100  | 0.0012        | -               |
+| 2.0    | 150  | 0.0003        | -               |
+| 2.6667 | 200  | 0.0003        | -               |
+| 3.3333 | 250  | 0.0002        | -               |
+| 4.0    | 300  | 0.0001        | -               |
+
+### Framework Versions
+- Python: 3.10.13
+- SetFit: 1.0.3
+- Sentence Transformers: 2.7.0
+- Transformers: 4.38.1
+- PyTorch: 2.1.2
+- Datasets: 2.19.0
+- Tokenizers: 0.15.2
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "sentence-transformers/all-roberta-large-v1",
+  "architectures": [
+    "RobertaModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.38.1",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

config_sentence_transformers.json

@@ -0,0 +1,9 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.0.0",
+    "transformers": "4.6.1",
+    "pytorch": "1.8.1"
+  },
+  "prompts": {},
+  "default_prompt_name": null
+}

config_setfit.json

@@ -0,0 +1,4 @@
+{
+  "labels": null,
+  "normalize_embeddings": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f9d8937185092da4ad87406e8efb306c4adafec466f0878b00016e95a99d359
+size 1421483904

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:653a6490d54676a06268211ad22fb93ef91390fe33b5d1dc097f6a92f3dafdac
+size 33639

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 256,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,64 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "max_length": 128,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "</s>",
+  "stride": 0,
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>"
+}