--- library_name: setfit tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer base_model: sentence-transformers/paraphrase-MiniLM-L6-v2 metrics: - accuracy widget: - text: What fabric has a comfortable feel and is suitable for people with sensitive skin? - text: What is the most recommended fabric for making outerwear that requires a blend of comfort and resilience? - text: What fabric has a fluid drape and is ideal for creating lightweight summer dresses? - text: Which fabric is best for creating versatile clothing items like casual shirts, blouses, and dresses in a periwinkle blue hue? - text: What kind of fabric is suitable for making form-fitting activewear like yoga pants and t-shirts? pipeline_tag: text-classification inference: true model-index: - name: SetFit with sentence-transformers/paraphrase-MiniLM-L6-v2 results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.3836898395721925 name: Accuracy --- # SetFit with sentence-transformers/paraphrase-MiniLM-L6-v2 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/paraphrase-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 128 tokens - **Number of Classes:** 75 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:-----------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Fabric ID 0462 | | | Fabric ID 0719_1 | | | Fabric ID 0862 | | | Fabric ID 0573_1 | | | Fabric ID 0455 | | | Fabric ID 0735 | | | Fabric ID 0863 | | | Fabric ID 0600 | | | Fabric ID 0736 | | | Fabric ID 0527_1 | | | Fabric ID 0453 | | | Fabric ID 0859 | | | Fabric ID 0745 | | | Fabric ID 0513 | | | Fabric ID 0873 | | | Fabric ID 0576_1 | | | Fabric ID 0456 | | | Fabric ID 0571 | | | Fabric ID 0462_1 | | | Fabric ID 0447 | | | Fabric ID 0645 | | | Fabric ID 0756 | | | Fabric ID 0612 | | | Fabric ID 0613 | | | Fabric ID 0768 | | | Fabric ID 0748 | | | Fabric ID 0528_1 | | | Fabric ID 0874 | | | Fabric ID 0742 | | | Fabric ID 0769 | | | Fabric ID 0770 | | | Fabric ID 0448 | | | Fabric ID 0725 | | | Fabric ID 0579 | | | Fabric ID 0522 | | | Fabric ID 0578 | | | Fabric ID 0526_1 | | | Fabric ID 0733 | | | Fabric ID 0575_1 | | | Fabric ID 0579_1 | | | Fabric ID 0722 | | | Fabric ID 0614 | | | Fabric ID 0575 | | | Fabric ID 0723 | | | Fabric ID 0598 | | | Fabric ID 0565 | | | Fabric ID 0512 | | | Fabric ID 0876 | | | Fabric ID 0856 | | | Fabric ID 0608 | | | Fabric ID 0573 | | | Fabric ID 0880 | | | Fabric ID 0450 | | | Fabric ID 0459 | | | Fabric ID 0564 | | | Fabric ID 0731 | | | Fabric ID 0578_1 | | | Fabric ID 0855 | | | Fabric ID 0772 | | | Fabric ID 0606 | | | Fabric ID 0596 | | | Fabric ID 0458 | | | Fabric ID 0523_1 | | | Fabric ID 0730 | | | Fabric ID 0449 | | | Fabric ID 0724 | | | Fabric ID 0734 | | | Fabric ID 0615 | | | Fabric ID 0869 | | | Fabric ID 0864 | | | Fabric ID 0616 | | | Fabric ID 0866 | | | Fabric ID 0601 | | | Fabric ID 0618 | | | Fabric ID 0773 | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.3837 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("Jazielinho/fabric_model_1") # Run inference preds = model("What fabric has a comfortable feel and is suitable for people with sensitive skin?") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 7 | 15.4858 | 30 | | Label | Training Sample Count | |:-----------------|:----------------------| | Fabric ID 0447 | 39 | | Fabric ID 0448 | 40 | | Fabric ID 0449 | 41 | | Fabric ID 0450 | 32 | | Fabric ID 0453 | 37 | | Fabric ID 0455 | 33 | | Fabric ID 0456 | 36 | | Fabric ID 0458 | 40 | | Fabric ID 0459 | 30 | | Fabric ID 0462 | 36 | | Fabric ID 0462_1 | 42 | | Fabric ID 0512 | 38 | | Fabric ID 0513 | 39 | | Fabric ID 0522 | 43 | | Fabric ID 0523_1 | 41 | | Fabric ID 0526_1 | 41 | | Fabric ID 0527_1 | 35 | | Fabric ID 0528_1 | 42 | | Fabric ID 0564 | 40 | | Fabric ID 0565 | 43 | | Fabric ID 0571 | 44 | | Fabric ID 0573 | 36 | | Fabric ID 0573_1 | 37 | | Fabric ID 0575 | 40 | | Fabric ID 0575_1 | 44 | | Fabric ID 0576_1 | 42 | | Fabric ID 0578 | 41 | | Fabric ID 0578_1 | 38 | | Fabric ID 0579 | 41 | | Fabric ID 0579_1 | 46 | | Fabric ID 0596 | 41 | | Fabric ID 0598 | 38 | | Fabric ID 0600 | 40 | | Fabric ID 0601 | 39 | | Fabric ID 0606 | 41 | | Fabric ID 0608 | 44 | | Fabric ID 0612 | 45 | | Fabric ID 0613 | 40 | | Fabric ID 0614 | 37 | | Fabric ID 0615 | 44 | | Fabric ID 0616 | 39 | | Fabric ID 0618 | 42 | | Fabric ID 0645 | 36 | | Fabric ID 0719_1 | 43 | | Fabric ID 0722 | 42 | | Fabric ID 0723 | 37 | | Fabric ID 0724 | 41 | | Fabric ID 0725 | 44 | | Fabric ID 0730 | 36 | | Fabric ID 0731 | 40 | | Fabric ID 0733 | 43 | | Fabric ID 0734 | 44 | | Fabric ID 0735 | 39 | | Fabric ID 0736 | 38 | | Fabric ID 0742 | 38 | | Fabric ID 0745 | 43 | | Fabric ID 0748 | 41 | | Fabric ID 0756 | 44 | | Fabric ID 0768 | 40 | | Fabric ID 0769 | 41 | | Fabric ID 0770 | 35 | | Fabric ID 0772 | 43 | | Fabric ID 0773 | 41 | | Fabric ID 0855 | 43 | | Fabric ID 0856 | 37 | | Fabric ID 0859 | 41 | | Fabric ID 0862 | 36 | | Fabric ID 0863 | 38 | | Fabric ID 0864 | 42 | | Fabric ID 0866 | 41 | | Fabric ID 0869 | 39 | | Fabric ID 0873 | 43 | | Fabric ID 0874 | 34 | | Fabric ID 0876 | 40 | | Fabric ID 0880 | 41 | ### Training Hyperparameters - batch_size: (256, 256) - num_epochs: (20, 20) - max_steps: -1 - sampling_strategy: undersampling - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:-------:|:--------:|:-------------:|:---------------:| | 0.0021 | 1 | 0.2732 | - | | 0.1040 | 50 | 0.2348 | - | | 0.2079 | 100 | 0.2277 | - | | 0.3119 | 150 | 0.2419 | - | | 0.4158 | 200 | 0.2401 | - | | 0.5198 | 250 | 0.2367 | - | | 0.6237 | 300 | 0.237 | - | | 0.7277 | 350 | 0.2372 | - | | 0.8316 | 400 | 0.2283 | - | | 0.9356 | 450 | 0.223 | - | | 1.0 | 481 | - | 0.207 | | 1.0395 | 500 | 0.2075 | - | | 1.1435 | 550 | 0.2162 | - | | 1.2474 | 600 | 0.1984 | - | | 1.3514 | 650 | 0.2173 | - | | 1.4553 | 700 | 0.2154 | - | | 1.5593 | 750 | 0.1912 | - | | 1.6632 | 800 | 0.2014 | - | | 1.7672 | 850 | 0.1866 | - | | 1.8711 | 900 | 0.1933 | - | | 1.9751 | 950 | 0.1821 | - | | 2.0 | 962 | - | 0.1863 | | 2.0790 | 1000 | 0.1607 | - | | 2.1830 | 1050 | 0.1544 | - | | 2.2869 | 1100 | 0.1624 | - | | 2.3909 | 1150 | 0.1586 | - | | 2.4948 | 1200 | 0.1445 | - | | 2.5988 | 1250 | 0.1662 | - | | 2.7027 | 1300 | 0.1515 | - | | 2.8067 | 1350 | 0.158 | - | | 2.9106 | 1400 | 0.1316 | - | | **3.0** | **1443** | **-** | **0.1824** | | 3.0146 | 1450 | 0.138 | - | | 3.1185 | 1500 | 0.1414 | - | | 3.2225 | 1550 | 0.1249 | - | | 3.3264 | 1600 | 0.1336 | - | | 3.4304 | 1650 | 0.1249 | - | | 3.5343 | 1700 | 0.1308 | - | | 3.6383 | 1750 | 0.1088 | - | | 3.7422 | 1800 | 0.122 | - | | 3.8462 | 1850 | 0.1029 | - | | 3.9501 | 1900 | 0.1065 | - | | 4.0 | 1924 | - | 0.1836 | | 4.0541 | 1950 | 0.1133 | - | | 4.1580 | 2000 | 0.1102 | - | | 4.2620 | 2050 | 0.1209 | - | | 4.3659 | 2100 | 0.1054 | - | | 4.4699 | 2150 | 0.0874 | - | | 4.5738 | 2200 | 0.0896 | - | | 4.6778 | 2250 | 0.1104 | - | | 4.7817 | 2300 | 0.0912 | - | | 4.8857 | 2350 | 0.0766 | - | | 4.9896 | 2400 | 0.0778 | - | | 5.0 | 2405 | - | 0.1952 | | 5.0936 | 2450 | 0.114 | - | | 5.1975 | 2500 | 0.0869 | - | | 5.3015 | 2550 | 0.0912 | - | | 5.4054 | 2600 | 0.103 | - | | 5.5094 | 2650 | 0.0748 | - | | 5.6133 | 2700 | 0.0911 | - | | 5.7173 | 2750 | 0.0721 | - | | 5.8212 | 2800 | 0.0964 | - | | 5.9252 | 2850 | 0.0712 | - | | 6.0 | 2886 | - | 0.1938 | | 6.0291 | 2900 | 0.0831 | - | | 6.1331 | 2950 | 0.0924 | - | | 6.2370 | 3000 | 0.0862 | - | | 6.3410 | 3050 | 0.0725 | - | | 6.4449 | 3100 | 0.0828 | - | | 6.5489 | 3150 | 0.0645 | - | | 6.6528 | 3200 | 0.0741 | - | | 6.7568 | 3250 | 0.0589 | - | | 6.8607 | 3300 | 0.075 | - | | 6.9647 | 3350 | 0.075 | - | | 7.0 | 3367 | - | 0.2016 | | 7.0686 | 3400 | 0.0893 | - | | 7.1726 | 3450 | 0.0727 | - | | 7.2765 | 3500 | 0.0669 | - | | 7.3805 | 3550 | 0.0702 | - | | 7.4844 | 3600 | 0.0636 | - | | 7.5884 | 3650 | 0.0605 | - | | 7.6923 | 3700 | 0.0707 | - | | 7.7963 | 3750 | 0.0597 | - | | 7.9002 | 3800 | 0.0577 | - | | 8.0 | 3848 | - | 0.2067 | | 8.0042 | 3850 | 0.0515 | - | | 8.1081 | 3900 | 0.0686 | - | | 8.2121 | 3950 | 0.0587 | - | | 8.3160 | 4000 | 0.057 | - | | 8.4200 | 4050 | 0.0693 | - | | 8.5239 | 4100 | 0.0812 | - | | 8.6279 | 4150 | 0.0592 | - | | 8.7318 | 4200 | 0.07 | - | | 8.8358 | 4250 | 0.064 | - | | 8.9397 | 4300 | 0.0503 | - | | 9.0 | 4329 | - | 0.2122 | | 9.0437 | 4350 | 0.0489 | - | | 9.1476 | 4400 | 0.0602 | - | | 9.2516 | 4450 | 0.0673 | - | | 9.3555 | 4500 | 0.0665 | - | | 9.4595 | 4550 | 0.0672 | - | | 9.5634 | 4600 | 0.07 | - | | 9.6674 | 4650 | 0.042 | - | | 9.7713 | 4700 | 0.0656 | - | | 9.8753 | 4750 | 0.0557 | - | | 9.9792 | 4800 | 0.0648 | - | | 10.0 | 4810 | - | 0.215 | | 10.0832 | 4850 | 0.0455 | - | | 10.1871 | 4900 | 0.0668 | - | | 10.2911 | 4950 | 0.0453 | - | | 10.3950 | 5000 | 0.0555 | - | | 10.4990 | 5050 | 0.0679 | - | | 10.6029 | 5100 | 0.0516 | - | | 10.7069 | 5150 | 0.0448 | - | | 10.8108 | 5200 | 0.0458 | - | | 10.9148 | 5250 | 0.0544 | - | | 11.0 | 5291 | - | 0.2172 | | 11.0187 | 5300 | 0.0453 | - | | 11.1227 | 5350 | 0.0442 | - | | 11.2266 | 5400 | 0.0396 | - | | 11.3306 | 5450 | 0.0507 | - | | 11.4345 | 5500 | 0.0515 | - | | 11.5385 | 5550 | 0.0503 | - | | 11.6424 | 5600 | 0.0521 | - | | 11.7464 | 5650 | 0.0551 | - | | 11.8503 | 5700 | 0.0572 | - | | 11.9543 | 5750 | 0.0604 | - | | 12.0 | 5772 | - | 0.2245 | | 12.0582 | 5800 | 0.0445 | - | | 12.1622 | 5850 | 0.0564 | - | | 12.2661 | 5900 | 0.0449 | - | | 12.3701 | 5950 | 0.0502 | - | | 12.4740 | 6000 | 0.0517 | - | | 12.5780 | 6050 | 0.0426 | - | | 12.6819 | 6100 | 0.0386 | - | | 12.7859 | 6150 | 0.0446 | - | | 12.8898 | 6200 | 0.0574 | - | | 12.9938 | 6250 | 0.0546 | - | | 13.0 | 6253 | - | 0.223 | | 13.0977 | 6300 | 0.0381 | - | | 13.2017 | 6350 | 0.047 | - | | 13.3056 | 6400 | 0.0425 | - | | 13.4096 | 6450 | 0.0445 | - | | 13.5135 | 6500 | 0.056 | - | | 13.6175 | 6550 | 0.0533 | - | | 13.7214 | 6600 | 0.0466 | - | | 13.8254 | 6650 | 0.0506 | - | | 13.9293 | 6700 | 0.0402 | - | | 14.0 | 6734 | - | 0.2238 | | 14.0333 | 6750 | 0.0375 | - | | 14.1372 | 6800 | 0.0447 | - | | 14.2412 | 6850 | 0.0584 | - | | 14.3451 | 6900 | 0.0348 | - | | 14.4491 | 6950 | 0.0459 | - | | 14.5530 | 7000 | 0.0465 | - | | 14.6570 | 7050 | 0.0421 | - | | 14.7609 | 7100 | 0.0537 | - | | 14.8649 | 7150 | 0.041 | - | | 14.9688 | 7200 | 0.0281 | - | | 15.0 | 7215 | - | 0.2247 | | 15.0728 | 7250 | 0.0431 | - | | 15.1767 | 7300 | 0.039 | - | | 15.2807 | 7350 | 0.0408 | - | | 15.3846 | 7400 | 0.048 | - | | 15.4886 | 7450 | 0.0354 | - | | 15.5925 | 7500 | 0.0626 | - | | 15.6965 | 7550 | 0.0396 | - | | 15.8004 | 7600 | 0.045 | - | | 15.9044 | 7650 | 0.0432 | - | | 16.0 | 7696 | - | 0.2246 | | 16.0083 | 7700 | 0.0385 | - | | 16.1123 | 7750 | 0.0368 | - | | 16.2162 | 7800 | 0.0628 | - | | 16.3202 | 7850 | 0.035 | - | | 16.4241 | 7900 | 0.0264 | - | | 16.5281 | 7950 | 0.0275 | - | | 16.6320 | 8000 | 0.0383 | - | | 16.7360 | 8050 | 0.0469 | - | | 16.8399 | 8100 | 0.0445 | - | | 16.9439 | 8150 | 0.0357 | - | | 17.0 | 8177 | - | 0.2268 | | 17.0478 | 8200 | 0.0456 | - | | 17.1518 | 8250 | 0.053 | - | | 17.2557 | 8300 | 0.0498 | - | | 17.3597 | 8350 | 0.0368 | - | | 17.4636 | 8400 | 0.0473 | - | | 17.5676 | 8450 | 0.0422 | - | | 17.6715 | 8500 | 0.0362 | - | | 17.7755 | 8550 | 0.0292 | - | | 17.8794 | 8600 | 0.0431 | - | | 17.9834 | 8650 | 0.0412 | - | | 18.0 | 8658 | - | 0.2276 | | 18.0873 | 8700 | 0.0655 | - | | 18.1913 | 8750 | 0.0405 | - | | 18.2952 | 8800 | 0.0455 | - | | 18.3992 | 8850 | 0.0324 | - | | 18.5031 | 8900 | 0.038 | - | | 18.6071 | 8950 | 0.0315 | - | | 18.7110 | 9000 | 0.0468 | - | | 18.8150 | 9050 | 0.0451 | - | | 18.9189 | 9100 | 0.032 | - | | 19.0 | 9139 | - | 0.2268 | | 19.0229 | 9150 | 0.0371 | - | | 19.1268 | 9200 | 0.0439 | - | | 19.2308 | 9250 | 0.0472 | - | | 19.3347 | 9300 | 0.0362 | - | | 19.4387 | 9350 | 0.0341 | - | | 19.5426 | 9400 | 0.036 | - | | 19.6466 | 9450 | 0.0382 | - | | 19.7505 | 9500 | 0.0288 | - | | 19.8545 | 9550 | 0.04 | - | | 19.9584 | 9600 | 0.0277 | - | | 20.0 | 9620 | - | 0.2277 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.3 - Sentence Transformers: 2.7.0 - Transformers: 4.40.1 - PyTorch: 2.2.1+cu121 - Datasets: 2.19.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```