Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,232 @@
+---
+base_model: sentence-transformers/paraphrase-mpnet-base-v2
+library_name: setfit
+metrics:
+- accuracy
+pipeline_tag: text-classification
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+widget:
+- text: Stocks plummeted on Friday as investors grew increasingly concerned about
+    a potential market correction. The Dow Jones Industrial Average fell 2.5% in a
+    single day, its worst performance since 2020. Analysts warn that the correction
+    could be a sign of a larger issue with the market, and that investors should be
+    prepared for a potential downturn. Meanwhile, the S&P 500 also declined, with
+    tech stocks leading the way down. The correction has left many investors wondering
+    if the market has reached a tipping point, and whether a bear market is on the
+    horizon.
+- text: The highly anticipated IPO of electric vehicle manufacturer, Rivian, is expected
+    to be a game-changer for the industry. With a strong order book and a proven track
+    record of innovation, Rivian is poised to disrupt the traditional auto sector.
+    As the company prepares to list on the NASDAQ, investors are eagerly awaiting
+    the opportunity to get in on the ground floor of this promising new player. With
+    a market capitalization of over $50 billion, Rivian is set to make a significant
+    impact on the stock market, and we believe it's a buy for long-term investors.
+- text: 'Just saw Q2 earnings release from @CompanyABC and I''m loving the 25% revenue
+    growth! Their expansion into new markets is paying off. Bullish on this stock
+    for the long haul #finance #stockmarket'
+- text: US GDP growth rate remains steady at 2.1% in Q1, meeting economist expectations.
+    The Federal Reserve's decision to keep interest rates unchanged is seen as a positive
+    sign for the economy, but some analysts warn of potential risks from rising inflation.
+- text: The latest GDP report is a clear indication that the economy is headed for
+    a recession. With a decline of 0.5% in the first quarter, it's only a matter of
+    time before businesses start to feel the pinch. The Fed's decision to raise interest
+    rates will only exacerbate the situation, making it even harder for consumers
+    to afford everyday essentials. I'm bearish on the market until we see a significant
+    turnaround in economic indicators.
+inference: true
+model-index:
+- name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.6520100502512562
+      name: Accuracy
+---
+
+# SetFit with sentence-transformers/paraphrase-mpnet-base-v2
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 3 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+|:------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 2     | <ul><li>"The US economy is expected to grow at a moderate pace in the coming quarters, with the Federal Reserve maintaining a neutral stance on interest rates. Analysts are predicting a slight increase in GDP, driven by consumer spending and business investment. However, some experts caution that global trade tensions and Brexit uncertainty could impact the country's economic outlook."</li><li>"Just took a closer look at the market trends and I'm not seeing anything that suggests a major shift in the near future. Volatility is still relatively low and the overall sentiment remains stable. Nothing to get too excited about, but nothing to worry about either."</li><li>'The US Securities and Exchange Commission (SEC) has announced a new set of guidelines for financial institutions to follow when reporting their quarterly earnings. The changes are aimed at providing more transparency and clarity for investors. The SEC stated that the new rules will take effect in the next quarter, and companies are expected to comply with the new guidelines by then. The move is seen as a positive step towards improving corporate governance and investor confidence.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+| 1     | <ul><li>"Just got out of the Q1 earnings call and I'm feeling bullish on $AAPL. Revenue growth is through the roof and their new product lineup is going to disrupt the market. Time to buy in! #Apple #StockMarket"</li><li>'The tech sector is poised for significant growth in the coming quarters, driven by the increasing adoption of cloud computing and artificial intelligence. With major players like Amazon and Microsoft leading the charge, investors can expect a substantial boost in stock prices. In fact, a recent report by Morgan Stanley suggests that the tech sector could see a 20% increase in revenue by the end of the year, making it an attractive space for investors looking to capitalize on the market expansion.'</li><li>"AI Breakthroughs Fuel Market Optimism: Experts Predict 30% Surge in Tech Stocks\n\nA recent report from leading AI research firm, DeepMind, has sent shockwaves through the financial community with its groundbreaking findings on neural network advancements. The breakthroughs, which include significant improvements in processing power and efficiency, are expected to have a profound impact on the tech industry. As a result, experts are predicting a 30% surge in tech stocks, with many analysts citing AI as the driving force behind this optimism. With the likes of Google and Microsoft already investing heavily in AI research, it's clear that this trend is here to stay. The Bullish sentiment is palpable, with many investors taking a long-term view on the potential of AI to drive growth and innovation."</li></ul>                                                                                                     |
+| 0     | <ul><li>"The latest GDP report is a major red flag for the economy, with a 0.5% contraction in Q2. This is a clear indication that the Federal Reserve's interest rate hikes are starting to bite, and it's only a matter of time before we see a recession. The market is already pricing in a 75% chance of a rate cut by the end of the year, but I'm not convinced that's enough to stem the tide. With the yield curve still inverted and consumer spending slowing, I'm bearish on the market's prospects for the rest of the year."</li><li>"Shares of Tesla (TSLA) plummeted 8.5% after the electric vehicle maker reported a surprise earnings miss due to a 27% decline in revenue. The company's revenue fell short of analyst expectations, casting a shadow over its growth prospects. This is the third consecutive quarter that Tesla has missed revenue estimates, raising concerns among investors about the company's ability to maintain its market share in a highly competitive EV market."</li><li>"Shares of struggling retailer, Sears Holdings, plummeted 12.5% on Tuesday after the company announced it would be closing an additional 150 stores, exacerbating concerns over its ability to stay afloat. The move is the latest in a series of efforts to stem the bleeding, but investors remain skeptical. With over $1.5 billion in debt and dwindling cash reserves, many are questioning whether Sears can avoid bankruptcy. The company's stock has lost nearly 75% of its value over the past year, leaving it on the brink of collapse. The news sent shockwaves through the retail sector, with other struggling retailers seeing their shares decline in response."</li></ul> |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 0.6520   |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("setfit_model_id")
+# Run inference
+preds = model("Just saw Q2 earnings release from @CompanyABC and I'm loving the 25% revenue growth! Their expansion into new markets is paying off. Bullish on this stock for the long haul #finance #stockmarket")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 27  | 75.1190 | 271 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| 0     | 15                    |
+| 1     | 16                    |
+| 2     | 11                    |
+
+### Training Hyperparameters
+- batch_size: (16, 16)
+- num_epochs: (5, 5)
+- max_steps: -1
+- sampling_strategy: oversampling
+- body_learning_rate: (2e-05, 1e-05)
+- head_learning_rate: 0.01
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: True
+
+### Training Results
+| Epoch   | Step    | Training Loss | Validation Loss |
+|:-------:|:-------:|:-------------:|:---------------:|
+| 0.0137  | 1       | 0.326         | -               |
+| 0.6849  | 50      | 0.0037        | -               |
+| 1.0     | 73      | -             | 0.0065          |
+| 1.3699  | 100     | 0.0006        | -               |
+| 2.0     | 146     | -             | 0.0047          |
+| 2.0548  | 150     | 0.0002        | -               |
+| 2.7397  | 200     | 0.0003        | -               |
+| 3.0     | 219     | -             | 0.0038          |
+| 3.4247  | 250     | 0.0001        | -               |
+| 4.0     | 292     | -             | 0.0038          |
+| 4.1096  | 300     | 0.0002        | -               |
+| 4.7945  | 350     | 0.0001        | -               |
+| **5.0** | **365** | **-**         | **0.0036**      |
+
+* The bold row denotes the saved checkpoint.
+### Framework Versions
+- Python: 3.9.19
+- SetFit: 1.1.0.dev0
+- Sentence Transformers: 3.0.1
+- Transformers: 4.39.0
+- PyTorch: 2.4.0
+- Datasets: 2.20.0
+- Tokenizers: 0.15.2
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "setfit/step_365",
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "torch_dtype": "float32",
+  "transformers_version": "4.39.0",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.39.0",
+    "pytorch": "2.4.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

config_setfit.json

@@ -0,0 +1,8 @@
+{
+  "normalize_embeddings": false,
+  "labels": [
+    "0",
+    "1",
+    "2"
+  ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3754648749baf4cfd3cd7d596948a47e3f132d6b6a6b1cb44fc4b9884efb9cc
+size 437967672

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4454ec63134a2bfa5992b549b96e587aede97ed9d138317e05d281a88e6b33ed
+size 19295

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,66 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "max_length": 512,
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "</s>",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}