Add SetFit model

1_Pooling/config.json

@@ -1,5 +1,5 @@
 {
-  "word_embedding_dimension": 384,
+  "word_embedding_dimension": 768,
   "pooling_mode_cls_token": false,
   "pooling_mode_mean_tokens": true,
   "pooling_mode_max_tokens": false,

README.md

@@ -8,25 +8,26 @@ tags:
 metrics:
 - accuracy
 widget:
-- text: This paper focuses on mining association rules between sets of items in large
-    databases, which can reveal interesting patterns and relationships among the data.
-- text: In this paper, the authors explore the economic concepts of fairness and retaliation
-    within the context of reciprocity, demonstrating how these principles shape market
-    behaviors and interactions.
-- text: Further research is needed to explore the applicability of the proposed model
-    to more complex multi-echelon inventory systems with additional features, such
-    as lead time variability and supplier reliability.
-- text: The NCEP/NCAR 40-Year Reanalysis Project provides retrospective atmospheric
-    data sets by assimilating observational data into a model, resulting in improved
-    estimates of historical weather patterns for meteorological research and applications.
-- text: This study aims to assess the accuracy of aerosol optical properties retrieved
-    from Aerosol Robotic Network (AERONET) Sun and sky radiance measurements using
-    ground-based reference data.
+- text: Further research is needed to develop more effective methods for the detection
+    and inhibition of ESBLs in clinical settings.
+- text: Although the phosphomolybdenum method presents high accuracy and precision
+    for vitamin E quantitation, its applicability to other antioxidants may require
+    further investigation.
+- text: The persistent inflammation observed in Interleukin-10-deficient mice provides
+    insight into the role of this cytokine in maintaining intestinal homeostasis and
+    highlights the potential implications for human diseases, such as inflammatory
+    bowel syndrome.
+- text: The proposed algorithms in this paper utilize Hamilton-Jacobi formulations
+    to calculate the front propagation speed, which depends on the curvature of the
+    front.
+- text: The IC50 values obtained from the semiautomated microdilution assay suggest
+    that artesunate and dihydroartemisinin exhibit comparable antimalarial activity
+    against the Plasmodium falciparum strains tested.
 pipeline_tag: text-classification
 inference: true
-base_model: sentence-transformers/paraphrase-MiniLM-L3-v2
+base_model: kaisugi/scitoricsbert
 model-index:
-- name: SetFit with sentence-transformers/paraphrase-MiniLM-L3-v2
+- name: SetFit with kaisugi/scitoricsbert
   results:
   - task:
       type: text-classification
@@ -37,13 +38,13 @@ model-index:
       split: test
     metrics:
     - type: accuracy
-      value: 0.7407692307692307
+      value: 0.8833333333333333
       name: Accuracy
 ---
 
-# SetFit with sentence-transformers/paraphrase-MiniLM-L3-v2
+# SetFit with kaisugi/scitoricsbert
 
-This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-MiniLM-L3-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L3-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [kaisugi/scitoricsbert](https://huggingface.co/kaisugi/scitoricsbert) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
 
 The model has been trained using an efficient few-shot learning technique that involves:
 
@@ -54,10 +55,10 @@ The model has been trained using an efficient few-shot learning technique that i
 
 ### Model Description
 - **Model Type:** SetFit
-- **Sentence Transformer body:** [sentence-transformers/paraphrase-MiniLM-L3-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L3-v2)
+- **Sentence Transformer body:** [kaisugi/scitoricsbert](https://huggingface.co/kaisugi/scitoricsbert)
 - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
-- **Maximum Sequence Length:** 128 tokens
-- **Number of Classes:** 13 classes
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 12 classes
 <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
 <!-- - **Language:** Unknown -->
 <!-- - **License:** Unknown -->
@@ -76,10 +77,9 @@ The model has been trained using an efficient few-shot learning technique that i
 | Hypothesis      | <ul><li>'Despite having average cholesterol levels, patients who received Pravastatin experienced a significant reduction in coronary events, suggesting a potential role for statins in preventing cardiovascular events beyond cholesterol level management in internal medicine.'</li><li>'This prospective observational study aimed to investigate the association between glycaemia levels and the risk of developing macrovascular and microvascular complications in individuals with type 2 diabetes, as previously identified in the UKPDS 35 study.'</li><li>'The results suggest that self-regulatory skills, particularly in the area of attention, significantly impact academic performance in elementary school students.'</li></ul>                                                                                    |
 | Implications    | <ul><li>'From 1995 to 1998, the UK Prospective Diabetes Study (UKPDS) 35 observed a significant association between higher glycaemia levels and increased risk of both macrovascular and microvascular complications in patients with type 2 diabetes.'</li><li>'The UKPDS 35 study provides robust evidence that every 1 mmol/L increase in HbA1c is associated with a 25% increased risk of macrovascular events and a 37% increased risk of microvascular complications in patients with type 2 diabetes, highlighting the importance of strict glycaemic control in internal medicine.'</li><li>"This study provides valuable insights into the early dynamics of the COVID-19 outbreak in Italy, contributing to the understanding of the disease's transmission patterns and impact on public health."</li></ul>                  |
 | Importance      | <ul><li>'Stroke and transient ischemic attack (TIA) are leading causes of long-term disability and mortality in internal medicine, with an estimated 15 million survivors worldwide.'</li><li>'The accurate assessment of insulin resistance and beta-cell function is crucial in the diagnosis and management of various metabolic disorders, including type 2 diabetes and metabolic syndrome.'</li><li>'The COVID-19 outbreak in Italy, which began in late February 2020, quickly became one of the most severe epidemic hotspots in Europe.'</li></ul>                                                                                                                                                                                                                                                                             |
-| Keywords        | <ul><li>'Pravastatin is a statin drug commonly used in the treatment of hypercholesterolemia, specifically to lower low-density lipoprotein (LDL) cholesterol levels and reduce the risk of cardiovascular events in internal medicine.'</li><li>'Self-regulation refers to the ability of students to manage their emotions, behavior, and cognitive processes to achieve optimal learning (Zimmerman & Kitsantas, 2005).'</li><li>'The proposed method utilizes deep convolutional neural networks to extract rich features from input images, enabling both object detection and semantic segmentation with high accuracy in the field of artificial intelligence.'</li></ul>                                                                                                                                                        |
 | Limitations     | <ul><li>'However, it is important to note that the Homeostasis Model Assessment (HOMA) index does not directly measure insulin sensitivity or β-cell function, but rather provides an estimate based on fasting plasma glucose and insulin concentrations.'</li><li>'Despite providing a useful estimate of insulin resistance and beta-cell function, the Homeostasis Model Assessment has limitations in its applicability to individuals with extreme glucose or insulin levels, as well as those with certain diseases such as liver disease or pregnancy.'</li><li>'Despite the large sample size and long follow-up period, the observational nature of the study limits the ability to establish causality between glycaemia and the observed complications in type 2 diabetes.'</li></ul>                                       |
 | Method          | <ul><li>'The study employed a randomized, double-blind, placebo-controlled design to investigate the effect of Pravastatin on coronary events in patients with average cholesterol levels.'</li><li>'Patients with a history of myocardial infarction and an average cholesterol level between 180 and 240 mg/dL were included in the study.'</li><li>'The study aimed to assess the impact of Pravastatin administration on the incidence of coronary events in internal medicine patients with average cholesterol levels.'</li></ul>                                                                                                                                                                                                                                                                                                 |
-| None            | <ul><li>'The study enrolled patients with a recent myocardial infarction and an average cholesterol level, who were then randomly assigned to receive either pravastatin or placebo.'</li><li>'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in the field of internal medicine.'</li><li>'This study aims to evaluate the effectiveness of the Homeostasis Model Assessment (HOMA) in estimating insulin resistance and pancreatic beta-cell function in internal medicine, offering valuable insights for the diagnosis and management of metabolic disorders.'</li></ul>                                                                                             |
+| None            | <ul><li>'Pravastatin is a statin drug commonly used in the treatment of hypercholesterolemia, specifically to lower low-density lipoprotein (LDL) cholesterol levels and reduce the risk of cardiovascular events in internal medicine.'</li><li>'The study enrolled patients with a recent myocardial infarction and an average cholesterol level, who were then randomly assigned to receive either pravastatin or placebo.'</li><li>'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in the field of internal medicine.'</li></ul>                                                                                                                                    |
 | Purpose         | <ul><li>'This study investigates the impact of Pravastatin on reducing coronary events in internal medicine patients with average cholesterol levels after a myocardial infarction.'</li><li>'This systematic review and meta-analysis aimed to assess the efficacy and safety of dual antiplatelet therapy with aspirin and clopidogrel in the secondary prevention of stroke and transient ischemic attack in internal medicine.'</li><li>'This study aims to evaluate the effectiveness of the Homeostasis Model Assessment (HOMA) in estimating insulin resistance and beta-cell function in internal medicine patients, addressing the need for a simple and widely applicable method for diagnosing and monitoring these conditions.'</li></ul>                                                                                   |
 | Reccomendations | <ul><li>'Further studies are needed to investigate the optimal duration of dual antiplatelet therapy in secondary prevention of stroke and transient ischemic attack, as well as the role of individual patient characteristics in determining the most effective treatment regimen.'</li><li>'Further research is warranted to explore the underlying mechanisms linking glycaemia to macrovascular and microvascular complications in type 2 diabetes, particularly in multi-ethnic populations.'</li><li>'Further studies are needed to investigate the potential role of IL-6 signaling in the prevention of bone loss in postmenopausal women.'</li></ul>                                                                                                                                                                          |
 | Result          | <ul><li>'Despite having average cholesterol levels, patients treated with Pravastatin did not experience a significant reduction in coronary events compared to the placebo group.'</li><li>'In interviews with patients who experienced a reduction in coronary events after Pravastatin treatment, themes included improved energy levels and increased confidence in managing their heart health.'</li><li>'The study found that Pravastatin significantly reduced the risk of coronary events in patients with average cholesterol levels, consistent with previous research suggesting that statins benefit a wider population beyond those with hypercholesterolemia.'</li></ul>                                                                                                                                                  |
@@ -90,7 +90,7 @@ The model has been trained using an efficient few-shot learning technique that i
 ### Metrics
 | Label   | Accuracy |
 |:--------|:---------|
-| **all** | 0.7408   |
+| **all** | 0.8833   |
 
 ## Uses
 
@@ -110,7 +110,7 @@ from setfit import SetFitModel
 # Download from the 🤗 Hub
 model = SetFitModel.from_pretrained("Corran/SciGenSetfit2")
 # Run inference
-preds = model("This paper focuses on mining association rules between sets of items in large databases, which can reveal interesting patterns and relationships among the data.")
+preds = model("Further research is needed to develop more effective methods for the detection and inhibition of ESBLs in clinical settings.")
 ```
 
 <!--
@@ -142,30 +142,29 @@ preds = model("This paper focuses on mining association rules between sets of it
 ### Training Set Metrics
 | Training set | Min | Median  | Max |
 |:-------------|:----|:--------|:----|
-| Word count   | 11  | 28.3123 | 71  |
+| Word count   | 11  | 28.3767 | 60  |
 
 | Label           | Training Sample Count |
 |:----------------|:----------------------|
-| Aims            | 200                   |
-| Background      | 200                   |
-| Hypothesis      | 200                   |
-| Implications    | 200                   |
-| Importance      | 200                   |
-| Keywords        | 200                   |
-| Limitations     | 200                   |
-| Method          | 200                   |
-| None            | 200                   |
-| Purpose         | 200                   |
-| Reccomendations | 200                   |
-| Result          | 200                   |
-| Uncertainty     | 200                   |
+| Aims            | 100                   |
+| Background      | 100                   |
+| Hypothesis      | 100                   |
+| Implications    | 100                   |
+| Importance      | 100                   |
+| Limitations     | 100                   |
+| Method          | 100                   |
+| None            | 100                   |
+| Purpose         | 100                   |
+| Reccomendations | 100                   |
+| Result          | 100                   |
+| Uncertainty     | 100                   |
 
 ### Training Hyperparameters
 - batch_size: (256, 256)
 - num_epochs: (1, 1)
 - max_steps: -1
 - sampling_strategy: oversampling
-- num_iterations: 40
+- num_iterations: 20
 - body_learning_rate: (2e-05, 1e-05)
 - head_learning_rate: 0.01
 - loss: CosineSimilarityLoss
@@ -181,23 +180,10 @@ preds = model("This paper focuses on mining association rules between sets of it
 ### Training Results
 | Epoch  | Step | Training Loss | Validation Loss |
 |:------:|:----:|:-------------:|:---------------:|
-| 0.0012 | 1    | 0.4201        | -               |
-| 0.0615 | 50   | 0.2562        | -               |
-| 0.1230 | 100  | 0.2334        | -               |
-| 0.1845 | 150  | 0.1974        | -               |
-| 0.2460 | 200  | 0.195         | -               |
-| 0.3075 | 250  | 0.1768        | -               |
-| 0.3690 | 300  | 0.146         | -               |
-| 0.4305 | 350  | 0.1541        | -               |
-| 0.4920 | 400  | 0.1647        | -               |
-| 0.5535 | 450  | 0.154         | -               |
-| 0.6150 | 500  | 0.1568        | -               |
-| 0.6765 | 550  | 0.1494        | -               |
-| 0.7380 | 600  | 0.1554        | -               |
-| 0.7995 | 650  | 0.1456        | -               |
-| 0.8610 | 700  | 0.1527        | -               |
-| 0.9225 | 750  | 0.1488        | -               |
-| 0.9840 | 800  | 0.1312        | -               |
+| 0.0053 | 1    | 0.2248        | -               |
+| 0.2660 | 50   | 0.1239        | -               |
+| 0.5319 | 100  | 0.1105        | -               |
+| 0.7979 | 150  | 0.0665        | -               |
 
 ### Framework Versions
 - Python: 3.10.12

config.json

@@ -1,26 +1,25 @@
 {
-  "_name_or_path": "/root/.cache/torch/sentence_transformers/sentence-transformers_paraphrase-MiniLM-L3-v2/",
+  "_name_or_path": "/root/.cache/torch/sentence_transformers/kaisugi_scitoricsbert",
   "architectures": [
     "BertModel"
   ],
   "attention_probs_dropout_prob": 0.1,
   "classifier_dropout": null,
-  "gradient_checkpointing": false,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,
-  "hidden_size": 384,
+  "hidden_size": 768,
   "initializer_range": 0.02,
-  "intermediate_size": 1536,
+  "intermediate_size": 3072,
   "layer_norm_eps": 1e-12,
   "max_position_embeddings": 512,
   "model_type": "bert",
   "num_attention_heads": 12,
-  "num_hidden_layers": 3,
+  "num_hidden_layers": 12,
   "pad_token_id": 0,
   "position_embedding_type": "absolute",
   "torch_dtype": "float32",
   "transformers_version": "4.36.2",
   "type_vocab_size": 2,
   "use_cache": true,
-  "vocab_size": 30522
+  "vocab_size": 31090
 }

config_sentence_transformers.json

@@ -1,7 +1,7 @@
 {
   "__version__": {
-    "sentence_transformers": "2.0.0",
-    "transformers": "4.7.0",
-    "pytorch": "1.9.0+cu102"
+    "sentence_transformers": "2.2.2",
+    "transformers": "4.36.2",
+    "pytorch": "2.1.0+cu121"
   }
 }

config_setfit.json

@@ -6,7 +6,6 @@
     "Hypothesis",
     "Implications",
     "Importance",
-    "Keywords",
     "Limitations",
     "Method",
     "None",

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:782421e8a8f86650f5c4c24184bb8cde66eb095e4f2bce737ad3508d1c844bd8
-size 69565312
+oid sha256:8e7b341f8e3b8fedb613b442497d31738e71ba61a696c7f6c6afb4d5f9356ed5
+size 439696224

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6790c7fffe6c2ab476607806d7b8ab06f8b147b2dce5a6a6eba84ea624ba05b8
-size 41647
+oid sha256:7e64694fc9f6f4206f0376da8dbcc8dafcf28de90cb1968623575fa434fb57f6
+size 75367

sentence_bert_config.json

@@ -1,4 +1,4 @@
 {
-  "max_seq_length": 128,
+  "max_seq_length": 512,
   "do_lower_case": false
 }

tokenizer_config.json

@@ -8,7 +8,7 @@
       "single_word": false,
       "special": true
     },
-    "100": {
+    "101": {
       "content": "[UNK]",
       "lstrip": false,
       "normalized": false,
@@ -16,7 +16,7 @@
       "single_word": false,
       "special": true
     },
-    "101": {
+    "102": {
       "content": "[CLS]",
       "lstrip": false,
       "normalized": false,
@@ -24,7 +24,7 @@
       "single_word": false,
       "special": true
     },
-    "102": {
+    "103": {
       "content": "[SEP]",
       "lstrip": false,
       "normalized": false,
@@ -32,7 +32,7 @@
       "single_word": false,
       "special": true
     },
-    "103": {
+    "104": {
       "content": "[MASK]",
       "lstrip": false,
       "normalized": false,
@@ -47,12 +47,9 @@
   "do_lower_case": true,
   "mask_token": "[MASK]",
   "max_length": 128,
-  "model_max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
   "never_split": null,
-  "pad_to_multiple_of": null,
   "pad_token": "[PAD]",
-  "pad_token_type_id": 0,
-  "padding_side": "right",
   "sep_token": "[SEP]",
   "stride": 0,
   "strip_accents": null,