Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,7 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false
+}

README.md

@@ -0,0 +1,229 @@
+---
+library_name: setfit
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+metrics:
+- accuracy
+widget:
+- text: Further research is needed to develop more effective methods for the detection
+    and inhibition of ESBLs in clinical settings.
+- text: Although the phosphomolybdenum method presents high accuracy and precision
+    for vitamin E quantitation, its applicability to other antioxidants may require
+    further investigation.
+- text: The persistent inflammation observed in Interleukin-10-deficient mice provides
+    insight into the role of this cytokine in maintaining intestinal homeostasis and
+    highlights the potential implications for human diseases, such as inflammatory
+    bowel syndrome.
+- text: The proposed algorithms in this paper utilize Hamilton-Jacobi formulations
+    to calculate the front propagation speed, which depends on the curvature of the
+    front.
+- text: The IC50 values obtained from the semiautomated microdilution assay suggest
+    that artesunate and dihydroartemisinin exhibit comparable antimalarial activity
+    against the Plasmodium falciparum strains tested.
+pipeline_tag: text-classification
+inference: true
+base_model: kaisugi/scitoricsbert
+model-index:
+- name: SetFit with kaisugi/scitoricsbert
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.8833333333333333
+      name: Accuracy
+---
+
+# SetFit with kaisugi/scitoricsbert
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [kaisugi/scitoricsbert](https://huggingface.co/kaisugi/scitoricsbert) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [kaisugi/scitoricsbert](https://huggingface.co/kaisugi/scitoricsbert)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 12 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label           | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+|:----------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Aims            | <ul><li>'This study aims to provide an in-depth analysis of the impact of Coronavirus Disease 2019 (COVID-19) on Italy, focusing on the early stages of the outbreak and the subsequent government response.'</li><li>'In this paper, we propose SegNet, a deep convolutional encoder-decoder architecture for real-time image segmentation.'</li><li>'This study aims to develop a mathematical model for analyzing genetic variation using restriction endonucleases.'</li></ul>                                                                                                                                                                                                                                                                                                                                                      |
+| Background      | <ul><li>'Previous studies have demonstrated that statins, including pravastatin, can reduce the risk of coronary events in patients with elevated cholesterol levels. However, the efficacy of pravastatin in patients with average cholesterol levels is less clear.'</li><li>'Previous studies have shown that statins, including pravastatin, can reduce the risk of coronary events in patients with elevated cholesterol levels. However, this study investigates the effect of pravastatin on patients with average cholesterol levels.'</li><li>'Previous studies have shown that statins, including pravastatin, can reduce the risk of coronary events in patients with elevated cholesterol levels. However, this trial investigates the effect of pravastatin on patients with average cholesterol levels.'</li></ul>        |
+| Hypothesis      | <ul><li>'Despite having average cholesterol levels, patients who received Pravastatin experienced a significant reduction in coronary events, suggesting a potential role for statins in preventing cardiovascular events beyond cholesterol level management in internal medicine.'</li><li>'This prospective observational study aimed to investigate the association between glycaemia levels and the risk of developing macrovascular and microvascular complications in individuals with type 2 diabetes, as previously identified in the UKPDS 35 study.'</li><li>'The results suggest that self-regulatory skills, particularly in the area of attention, significantly impact academic performance in elementary school students.'</li></ul>                                                                                    |
+| Implications    | <ul><li>'From 1995 to 1998, the UK Prospective Diabetes Study (UKPDS) 35 observed a significant association between higher glycaemia levels and increased risk of both macrovascular and microvascular complications in patients with type 2 diabetes.'</li><li>'The UKPDS 35 study provides robust evidence that every 1 mmol/L increase in HbA1c is associated with a 25% increased risk of macrovascular events and a 37% increased risk of microvascular complications in patients with type 2 diabetes, highlighting the importance of strict glycaemic control in internal medicine.'</li><li>"This study provides valuable insights into the early dynamics of the COVID-19 outbreak in Italy, contributing to the understanding of the disease's transmission patterns and impact on public health."</li></ul>                  |
+| Importance      | <ul><li>'Stroke and transient ischemic attack (TIA) are leading causes of long-term disability and mortality in internal medicine, with an estimated 15 million survivors worldwide.'</li><li>'The accurate assessment of insulin resistance and beta-cell function is crucial in the diagnosis and management of various metabolic disorders, including type 2 diabetes and metabolic syndrome.'</li><li>'The COVID-19 outbreak in Italy, which began in late February 2020, quickly became one of the most severe epidemic hotspots in Europe.'</li></ul>                                                                                                                                                                                                                                                                             |
+| Limitations     | <ul><li>'However, it is important to note that the Homeostasis Model Assessment (HOMA) index does not directly measure insulin sensitivity or β-cell function, but rather provides an estimate based on fasting plasma glucose and insulin concentrations.'</li><li>'Despite providing a useful estimate of insulin resistance and beta-cell function, the Homeostasis Model Assessment has limitations in its applicability to individuals with extreme glucose or insulin levels, as well as those with certain diseases such as liver disease or pregnancy.'</li><li>'Despite the large sample size and long follow-up period, the observational nature of the study limits the ability to establish causality between glycaemia and the observed complications in type 2 diabetes.'</li></ul>                                       |
+| Method          | <ul><li>'The study employed a randomized, double-blind, placebo-controlled design to investigate the effect of Pravastatin on coronary events in patients with average cholesterol levels.'</li><li>'Patients with a history of myocardial infarction and an average cholesterol level between 180 and 240 mg/dL were included in the study.'</li><li>'The study aimed to assess the impact of Pravastatin administration on the incidence of coronary events in internal medicine patients with average cholesterol levels.'</li></ul>                                                                                                                                                                                                                                                                                                 |
+| None            | <ul><li>'Pravastatin is a statin drug commonly used in the treatment of hypercholesterolemia, specifically to lower low-density lipoprotein (LDL) cholesterol levels and reduce the risk of cardiovascular events in internal medicine.'</li><li>'The study enrolled patients with a recent myocardial infarction and an average cholesterol level, who were then randomly assigned to receive either pravastatin or placebo.'</li><li>'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in the field of internal medicine.'</li></ul>                                                                                                                                    |
+| Purpose         | <ul><li>'This study investigates the impact of Pravastatin on reducing coronary events in internal medicine patients with average cholesterol levels after a myocardial infarction.'</li><li>'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in internal medicine.'</li><li>'This study aims to evaluate the effectiveness of the Homeostasis Model Assessment (HOMA) in estimating insulin resistance and beta-cell function in internal medicine patients, addressing the need for a simple and widely applicable method for diagnosing and monitoring these conditions.'</li></ul>                                                                                   |
+| Reccomendations | <ul><li>'Further studies are needed to investigate the optimal duration of dual antiplatelet therapy in secondary prevention of stroke and transient ischemic attack, as well as the role of individual patient characteristics in determining the most effective treatment regimen.'</li><li>'Further research is warranted to explore the underlying mechanisms linking glycaemia to macrovascular and microvascular complications in type 2 diabetes, particularly in multi-ethnic populations.'</li><li>'Further studies are needed to investigate the potential role of IL-6 signaling in the prevention of bone loss in postmenopausal women.'</li></ul>                                                                                                                                                                          |
+| Result          | <ul><li>'Despite having average cholesterol levels, patients treated with Pravastatin did not experience a significant reduction in coronary events compared to the placebo group.'</li><li>'In interviews with patients who experienced a reduction in coronary events after Pravastatin treatment, themes included improved energy levels and increased confidence in managing their heart health.'</li><li>'The study found that Pravastatin significantly reduced the risk of coronary events in patients with average cholesterol levels, consistent with previous research suggesting that statins benefit a wider population beyond those with hypercholesterolemia.'</li></ul>                                                                                                                                                  |
+| Uncertainty     | <ul><li>'Despite the widespread use of pravastatin in post-myocardial infarction patients with average cholesterol levels, the evidence regarding its impact on coronary events remains inconclusive and sometimes contradictory.'</li><li>'Despite the findings of this study showing a reduction in coronary events with Pravastatin use in patients with average cholesterol levels, contrasting evidence exists suggesting no significant benefit in similar patient populations (Miller et al., 2018).'</li><li>'Despite the proven benefits of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of cardiovascular events, particularly in coronary artery disease, there is a paucity of data specifically addressing its use in stroke or transient ischemic attack (TIA) patients.'</li></ul> |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 0.8833   |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("Corran/SciGenSetfit4")
+# Run inference
+preds = model("Further research is needed to develop more effective methods for the detection and inhibition of ESBLs in clinical settings.")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 11  | 28.3767 | 60  |
+
+| Label           | Training Sample Count |
+|:----------------|:----------------------|
+| Aims            | 100                   |
+| Background      | 100                   |
+| Hypothesis      | 100                   |
+| Implications    | 100                   |
+| Importance      | 100                   |
+| Limitations     | 100                   |
+| Method          | 100                   |
+| None            | 100                   |
+| Purpose         | 100                   |
+| Reccomendations | 100                   |
+| Result          | 100                   |
+| Uncertainty     | 100                   |
+
+### Training Hyperparameters
+- batch_size: (256, 256)
+- num_epochs: (1, 1)
+- max_steps: -1
+- sampling_strategy: oversampling
+- num_iterations: 20
+- body_learning_rate: (2e-05, 1e-05)
+- head_learning_rate: 0.01
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0053 | 1    | 0.2248        | -               |
+| 0.2660 | 50   | 0.1239        | -               |
+| 0.5319 | 100  | 0.1105        | -               |
+| 0.7979 | 150  | 0.0665        | -               |
+
+### Framework Versions
+- Python: 3.10.12
+- SetFit: 1.0.3
+- Sentence Transformers: 2.2.2
+- Transformers: 4.36.2
+- PyTorch: 2.1.0+cu121
+- Datasets: 2.16.1
+- Tokenizers: 0.15.0
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "/root/.cache/torch/sentence_transformers/kaisugi_scitoricsbert",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.36.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 31090
+}

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.2.2",
+    "transformers": "4.36.2",
+    "pytorch": "2.1.0+cu121"
+  }
+}

config_setfit.json

@@ -0,0 +1,17 @@
+{
+  "normalize_embeddings": false,
+  "labels": [
+    "Aims",
+    "Background",
+    "Hypothesis",
+    "Implications",
+    "Importance",
+    "Limitations",
+    "Method",
+    "None",
+    "Purpose",
+    "Reccomendations",
+    "Result",
+    "Uncertainty"
+  ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e7b341f8e3b8fedb613b442497d31738e71ba61a696c7f6c6afb4d5f9356ed5
+size 439696224

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e64694fc9f6f4206f0376da8dbcc8dafcf28de90cb1968623575fa434fb57f6
+size 75367

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,61 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 1000000000000000019884624838656,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}