--- library_name: setfit tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer datasets: - konsman/setfit-messages-optimized metrics: - f1 - accuracy widget: - text: Tomato sauce is acidic and causes problems with my reflux. That, in turn, irritates the vagus nerve and may bring on arrthymia. I can't eat tomato soup anymore. It could be a trigger for that reason. - text: pednisone is synthetic cortisol hormone naturally produced by our adrenal glands , a powerful anti inflamatory , it works by reducing swelling . I recommend reading the book "adrenal fatigue - the 21st century health syndrome" , the doc says all allergies , frequent respiratory tract infections and asthma have an underlying cause that is adrenal fatigue.. - text: 'You may want to read about Nigella sativa. It is helpful for many conditions, and studies have been done showing it to be beneficial at reducing inflammation of ulcerative colitis. It is also generally good for preventing many diseases, including cancer. Also hemorrhoids. ' - text: Sorry forgot to say that unfortunately after this problem that made me let sports and with the anxiety meds . I am now 83 kg - text: 6 months pregnant had an abnormal pap, doctor did a biopsy and came back as cis what is this how serious and what's the cause? I have to have a leep after my son comes, what does this entail? Doc not good at explaining anything pipeline_tag: text-classification inference: false base_model: sentence-transformers/paraphrase-mpnet-base-v2 model-index: - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2 results: - task: type: text-classification name: Text Classification dataset: name: konsman/setfit-messages-optimized type: konsman/setfit-messages-optimized split: test metrics: - type: f1 value: 0.6896901980700864 name: F1 - type: accuracy value: 0.3403755868544601 name: Accuracy --- # SetFit with sentence-transformers/paraphrase-mpnet-base-v2 This is a [SetFit](https://github.com/huggingface/setfit) model trained on the [konsman/setfit-messages-optimized](https://huggingface.co/datasets/konsman/setfit-messages-optimized) dataset that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A MultiOutputClassifier instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) - **Classification head:** a MultiOutputClassifier instance - **Maximum Sequence Length:** 512 tokens - **Training Dataset:** [konsman/setfit-messages-optimized](https://huggingface.co/datasets/konsman/setfit-messages-optimized) ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ## Evaluation ### Metrics | Label | F1 | Accuracy | |:--------|:-------|:---------| | **all** | 0.6897 | 0.3404 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("konsman/setfit-messages-multilabel-example") # Run inference preds = model("Sorry forgot to say that unfortunately after this problem that made me let sports and with the anxiety meds . I am now 83 kg") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:---------|:----| | Word count | 5 | 110.2344 | 469 | ### Training Hyperparameters - batch_size: (8, 8) - num_epochs: (2, 2) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 5 - body_learning_rate: (2e-05, 2e-05) - head_learning_rate: 2e-05 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: False ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:------:|:----:|:-------------:|:---------------:| | 0.0031 | 1 | 0.3209 | - | | 0.1562 | 50 | 0.1823 | - | | 0.3125 | 100 | 0.1003 | - | | 0.4688 | 150 | 0.1774 | - | | 0.625 | 200 | 0.0832 | - | | 0.7812 | 250 | 0.0828 | - | | 0.9375 | 300 | 0.0721 | - | | 1.0938 | 350 | 0.1331 | - | | 1.25 | 400 | 0.1215 | - | | 1.4062 | 450 | 0.1494 | - | | 1.5625 | 500 | 0.0444 | - | | 1.7188 | 550 | 0.0688 | - | | 1.875 | 600 | 0.1033 | - | | 0.0125 | 1 | 0.0508 | - | | 0.625 | 50 | 0.0793 | - | | 1.25 | 100 | 0.081 | - | | 1.875 | 150 | 0.1367 | - | ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.2 - Sentence Transformers: 2.2.2 - Transformers: 4.35.2 - PyTorch: 2.1.0+cu121 - Datasets: 2.16.1 - Tokenizers: 0.15.0 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```