Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,7 @@
+{
+  "word_embedding_dimension": 384,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false
+}

README.md

@@ -0,0 +1,260 @@
+---
+library_name: setfit
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+metrics:
+- accuracy
+widget:
+- text: The aim of this study was to investigate the effect of diets of extreme macronutrient
+    composition on DIT under near physiological conditions in a respiration chamber
+    over the duration of a full day.
+- text: It can be seen from the figure that the blue boundaries divide the spectrum
+    into too many areas.
+- text: It may be the case that the seller commits to selling the product to the buyer
+    immediately after checking the order.
+- text: These subjects were excluded from the study.
+- text: While the chemical shift predictions that are used always have some level
+    of error, a key benefit of this approach is that individual errors of large magnitude
+    are easily identified and tolerated due to redundancy in the network of moving
+    peaks.
+pipeline_tag: text-classification
+inference: true
+base_model: sentence-transformers/all-MiniLM-L6-v2
+model-index:
+- name: SetFit with sentence-transformers/all-MiniLM-L6-v2
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.9755555555555555
+      name: Accuracy
+---
+
+# SetFit with sentence-transformers/all-MiniLM-L6-v2
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 256 tokens
+- **Number of Classes:** 9 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+|:------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 1     | <ul><li>'As the results indicate, significant differences were found between the experimental group and the control group concerning the characteristics of the exploration process.'</li><li>'No significant differences were found between fallers and non-fallers with respect to height, weight, or age.'</li><li>'There was a significant difference between the 5% calcium hypochlorite group and the other groups (P<0.001).'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+| 2     | <ul><li>'Our study was also limited by the lack of studies that reported age and gender-specific incidence for morbidity and mortality.'</li><li>'And while quiet stance was examined here, it is important to emphasize that the use of perturbations have provided great insight into those at risk of falling, and future prospective trials which incorporate more sophisticated assessment of fall risk are certain to provide critical information on the reactive mechanics of stability and the effects of age-related degradation on individual balance strategies [25, 26] .Another limitation of this study is the dependence of self-reporting of falls, the key parameter used to stratify the elderly groups into those with recent fall history or those with a limited history of falls.'</li><li>"Because a patient's immigration status is not recorded concomitantly with hospital resource use in any hospital, state, or federal database, it is not currently possible to isolate charity care and bad debt expenditures on An additional complicating factor is the possibility that, as a result of PRWORA, hospitals may provide and bill for services as emergency services that previously were categorized as nonemergency services in order to secure Medicaid payment."</li></ul> |
+| 3     | <ul><li>'An 3-(4,5-dimethylthiazol-2yl)-2,5diphenyl tetrazolium bromide assay was used to evaluate the cytotoxicity of polyplexes at a series of N/P ratios in C6 and Hep G2 cells cultured in DMEM (with 10% fetal bovine serum) according to the methods described in our previous studies.'</li><li>'A multivariate analysis using logistic regression was used to evaluate the independent role of each covariate in hospital mortality.'</li><li>'Different methods have been used in the literature for implementing and updating the routing tables using the ant approach such as AntNet [1] .'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+| 4     | <ul><li>'The results of this study indicate that only the right GVS interfered with mental transformation.'</li><li>'The goal of this work is to explore the effects of general relativity on TDEs occurring in eccentric nuclear disks, and to quantify the distribution of orbital elements of TDEs that originate in eccentric nuclear disks.'</li><li>'Our results may have a number of important implications to the astrophysics of relativistic plasma in general and that of PWN in particular.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+| 5     | <ul><li>'The gel retardation results of polymer/pDNA complexes with increasing N/P ratios are shown in Figure 1 .'</li><li>'In line with this, it has been suggested that the drift occurs only when the observed rubber hand is congruent in terms of posture and identity with the participants unseen hand (Tsakiris and Haggard, 2005) .'</li><li>'Mortality rates have been found to be high.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+| 6     | <ul><li>'In order to use the information on prior falls in the prediction algorithm, elderly subjects were divided into two groups; those with a record of self-reported recent falls (n = 24; 14.9% of total elderly group) and those who had reported no falls in the prior sixmonth period (n = 137; 85.1% of total elderly group).'</li><li>"Semi-structured interviews were conducted with four 'custodians' (people working in locations where devices were deployed)."</li><li>'Patients who had previously undergone spinal surgery were excluded from the study.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+| 7     | <ul><li>'Then, the cells were incubated for 4 h, and fresh media were added to the culture for another 20 h. Then, 10 μl of sterile, filtered 3-(4,5-dimethylthiazol-2yl)-2,5diphenyl tetrazolium bromide solution in phosphate-buffered saline (PBS) (5 mg ml −1 ) was added to each well.'</li><li>'One of the key problems in this area is the identification of influential users, by targeting whom certain desirable outcomes can be achieved.'</li><li>'The paper proceeds as follows.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+| 8     | <ul><li>'The main aim of this paper is to present astrophysical parameters such as reddening, distance and age of Be 8 from four colour indices, (B − V ) , (V − I) , (R − I) and (G BP -G RP ) obtained from deep CCD U BV RI and Gaia photometries.'</li><li>'A key finding of the present study was that the rapid increase in GATA4 binding activity in cardiac nuclear extracts in response to pressure overload is mediated by ET-1 but not Ang II.'</li><li>'Section II of this paper provides an overview of the Bosch DCMG system and its components.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+| 9     | <ul><li>'These results provide additional support for an activating role for H3K4me3 and a silencing role for H3K27me3 as leaves age.'</li><li>'Based on this result, it may be the case that the rate of apoptosis increases after day 5. in a previous study, mirnas were found to regulate cell proliferation, cell cycle progression and migration by altering the expressions of various factors, such as MalaT1 (48) .'</li><li>'It is therefore likely that the efforts put in by many groups to unravel the spatial regulation of the bAR system will be relevant for the understanding of human disease.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 0.9756   |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("Corran/SciFunctions")
+# Run inference
+preds = model("These subjects were excluded from the study.")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 5   | 26.0891 | 245 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| 1     | 450                   |
+| 2     | 450                   |
+| 3     | 450                   |
+| 4     | 450                   |
+| 5     | 450                   |
+| 6     | 450                   |
+| 7     | 450                   |
+| 8     | 450                   |
+| 9     | 450                   |
+
+### Training Hyperparameters
+- batch_size: (75, 75)
+- num_epochs: (1, 1)
+- max_steps: -1
+- sampling_strategy: oversampling
+- num_iterations: 20
+- body_learning_rate: (2e-05, 2e-05)
+- head_learning_rate: 2e-05
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0005 | 1    | 0.3763        | -               |
+| 0.0231 | 50   | 0.317         | -               |
+| 0.0463 | 100  | 0.2252        | -               |
+| 0.0694 | 150  | 0.189         | -               |
+| 0.0926 | 200  | 0.1505        | -               |
+| 0.1157 | 250  | 0.105         | -               |
+| 0.1389 | 300  | 0.1024        | -               |
+| 0.1620 | 350  | 0.0867        | -               |
+| 0.1852 | 400  | 0.0659        | -               |
+| 0.2083 | 450  | 0.0532        | -               |
+| 0.2315 | 500  | 0.0366        | -               |
+| 0.2546 | 550  | 0.0622        | -               |
+| 0.2778 | 600  | 0.0241        | -               |
+| 0.3009 | 650  | 0.0315        | -               |
+| 0.3241 | 700  | 0.025         | -               |
+| 0.3472 | 750  | 0.0412        | -               |
+| 0.3704 | 800  | 0.0274        | -               |
+| 0.3935 | 850  | 0.0203        | -               |
+| 0.4167 | 900  | 0.0302        | -               |
+| 0.4398 | 950  | 0.0152        | -               |
+| 0.4630 | 1000 | 0.0103        | -               |
+| 0.4861 | 1050 | 0.0102        | -               |
+| 0.5093 | 1100 | 0.0208        | -               |
+| 0.5324 | 1150 | 0.0168        | -               |
+| 0.5556 | 1200 | 0.0158        | -               |
+| 0.5787 | 1250 | 0.0045        | -               |
+| 0.6019 | 1300 | 0.014         | -               |
+| 0.625  | 1350 | 0.0061        | -               |
+| 0.6481 | 1400 | 0.0125        | -               |
+| 0.6713 | 1450 | 0.0048        | -               |
+| 0.6944 | 1500 | 0.0042        | -               |
+| 0.7176 | 1550 | 0.0055        | -               |
+| 0.7407 | 1600 | 0.0058        | -               |
+| 0.7639 | 1650 | 0.0032        | -               |
+| 0.7870 | 1700 | 0.0041        | -               |
+| 0.8102 | 1750 | 0.0042        | -               |
+| 0.8333 | 1800 | 0.0018        | -               |
+| 0.8565 | 1850 | 0.0094        | -               |
+| 0.8796 | 1900 | 0.0096        | -               |
+| 0.9028 | 1950 | 0.0043        | -               |
+| 0.9259 | 2000 | 0.003         | -               |
+| 0.9491 | 2050 | 0.0029        | -               |
+| 0.9722 | 2100 | 0.0016        | -               |
+| 0.9954 | 2150 | 0.0084        | -               |
+
+### Framework Versions
+- Python: 3.10.12
+- SetFit: 1.0.1
+- Sentence Transformers: 2.2.2
+- Transformers: 4.35.2
+- PyTorch: 2.1.0+cu121
+- Datasets: 2.16.1
+- Tokenizers: 0.15.0
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "/root/.cache/torch/sentence_transformers/sentence-transformers_all-MiniLM-L6-v2/",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.35.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.0.0",
+    "transformers": "4.6.1",
+    "pytorch": "1.8.1"
+  }
+}

config_setfit.json

@@ -0,0 +1,4 @@
+{
+  "normalize_embeddings": false,
+  "labels": null
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6eb2755d97ebd3d807dbdc8159e65ddf64d1f1aad05088a3a0f3a77b94dc6565
+size 90864192

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95749c9bde6ef957c53d04db9cae6d10e02dfeaa39bc14d9f8a99e8dc4eb0cda
+size 28623

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 256,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,64 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}