--- library_name: setfit tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer metrics: - '0' - '1' - '2' - accuracy - macro avg - weighted avg widget: - text: 'Devi Sri Prasad : il aurait dû envoyer Pushpa aux Oscars Pour le compositeur de musique Devi Sri Prasad, alias DSP, le fait d''avoir remporté le National Film Award du meilleur directeur musical (chansons) pour Pushpa : The Rise (2021) "signifie beaucoup" et il est "reconnaissant de cet honneur"' - text: 'Parmi les nouveautés qui valent la peine d''être écoutées, citons les albums d''Offset et de Troye Sivan, un groupe de soi-disant méchants d''émissions de télé-réalité et de compétition qui s''affrontent dans un nouveau jeu de télé-réalité intitulé "House of Villains", et le jeu vidéo Forza Motorsport qui vous offre la possibilité de conduire plus de 500 voitures qui sont toutes plus sexy que celles qui encombrent votre entrée Foxx, Offset, Musk, retour de Frasier Jamie Foxx et Tommy Lee Jones à la tête du drame judiciaire "The Burial" et le retour de "Frasier" avec Kelsey Grammer font partie des nouveautés télévisuelles, cinématographiques et musicales qui arriveront sur un appareil près de chez vous Elle marque un nouveau chapitre de ma vie", a déclaré Offset dans un communiqué. "Cette œuvre est une guérison pour moi et une lettre à mes fans et à ceux qui me soutiennent "Set It Off" est caractéristique d''Offset - énergique, empathique, avec une liste impressionnante de collaborateurs, comme en témoigne "Jealousy", qui met en scène sa femme Cardi B et s''inspire d''un morceau de Three 6 Mafia "Set It Off", le deuxième album solo d''Offset, membre de Migos, et son premier album complet depuis la mort de son compagnon de groupe et cousin Takeoff, sort vendredi' - text: 'Bluebird, une filiale de Pan Macmillan, a publié les titres de Russell Brand, notamment Recovery : Freedom from Our Addictions et Mentors : How to Help and Be Helped ces dernières années Les spectacles de Russell Brand sont reportés à la suite d''allégations d''agression Les promoteurs du spectacle de standup de l''humoriste Russell Brand ont annulé ses représentations pour les dix prochains jours, tandis que son éditeur a annoncé qu''il mettait en "pause" tous ses futurs projets de livres après la publication, le week-end dernier, d''allégations de viol et d''agression sexuelle à son encontre' - text: 'Italie : Meloni admet qu''elle espérait faire "mieux" en matière d''immigration alors que les chiffres montent en flèche Le Premier ministre italien, Giorgia Meloni, a admis qu''elle avait espéré faire "mieux" pour contrôler l''immigration irrégulière, qui a considérablement augmenté depuis la victoire électorale historique de son parti d''extrême droite il y a un an' - text: 'Le juge, Andrew Hanen, du tribunal de district de Houston, a estimé que le président Barack Obama avait outrepassé ses pouvoirs lorsqu''il avait créé le programme DACA (Deferred Action for Childhood Arrivals) par voie d''action exécutive en 2012. Cette décision est le dernier rebondissement d''une saga judiciaire de cinq ans qui a laissé le programme et ses bénéficiaires, connus sous le nom de Dreamers, en suspens' pipeline_tag: text-classification inference: true base_model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2 model-index: - name: SetFit with sentence-transformers/paraphrase-multilingual-mpnet-base-v2 results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: '0' value: precision: 0.9702970297029703 recall: 0.9671052631578947 f1-score: 0.9686985172981877 support: 912 name: '0' - type: '1' value: precision: 0.9698451507742462 recall: 0.9754098360655737 f1-score: 0.9726195341234164 support: 1220 name: '1' - type: '2' value: precision: 0.9900442477876106 recall: 0.98568281938326 f1-score: 0.987858719646799 support: 908 name: '2' - type: accuracy value: 0.9759868421052632 name: Accuracy - type: macro avg value: precision: 0.976728809421609 recall: 0.9760659728689095 f1-score: 0.9763922570228011 support: 3040 name: Macro Avg - type: weighted avg value: precision: 0.9760138657976447 recall: 0.9759868421052632 f1-score: 0.9759949331729633 support: 3040 name: Weighted Avg --- # SetFit with sentence-transformers/paraphrase-multilingual-mpnet-base-v2 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 128 tokens - **Number of Classes:** 3 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | pos | | | obj | | | neg | | ## Evaluation ### Metrics | Label | 0 | 1 | 2 | Accuracy | Macro Avg | Weighted Avg | |:--------|:----------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------|:---------|:----------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------| | **all** | {'precision': 0.9702970297029703, 'recall': 0.9671052631578947, 'f1-score': 0.9686985172981877, 'support': 912} | {'precision': 0.9698451507742462, 'recall': 0.9754098360655737, 'f1-score': 0.9726195341234164, 'support': 1220} | {'precision': 0.9900442477876106, 'recall': 0.98568281938326, 'f1-score': 0.987858719646799, 'support': 908} | 0.9760 | {'precision': 0.976728809421609, 'recall': 0.9760659728689095, 'f1-score': 0.9763922570228011, 'support': 3040} | {'precision': 0.9760138657976447, 'recall': 0.9759868421052632, 'f1-score': 0.9759949331729633, 'support': 3040} | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("mogaio/pr_ebsa_fr_tran_merged25_e1_beginning_offsets_10") # Run inference preds = model("Devi Sri Prasad : il aurait dû envoyer Pushpa aux Oscars Pour le compositeur de musique Devi Sri Prasad, alias DSP, le fait d'avoir remporté le National Film Award du meilleur directeur musical (chansons) pour Pushpa : The Rise (2021) \"signifie beaucoup\" et il est \"reconnaissant de cet honneur\"") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:---------|:-----| | Word count | 1 | 243.9997 | 2071 | | Label | Training Sample Count | |:------|:----------------------| | neg | 912 | | obj | 1220 | | pos | 908 | ### Training Hyperparameters - batch_size: (8, 8) - num_epochs: (10, 10) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 1 - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:-------:|:--------:|:-------------:|:---------------:| | 0.0013 | 1 | 0.3488 | - | | 0.0658 | 50 | 0.3523 | - | | 0.1316 | 100 | 0.1781 | - | | 0.1974 | 150 | 0.2468 | - | | 0.2632 | 200 | 0.2954 | - | | 0.3289 | 250 | 0.1685 | - | | 0.3947 | 300 | 0.1585 | - | | 0.4605 | 350 | 0.2223 | - | | 0.5263 | 400 | 0.2515 | - | | 0.5921 | 450 | 0.301 | - | | 0.6579 | 500 | 0.3206 | - | | 0.7237 | 550 | 0.3041 | - | | 0.7895 | 600 | 0.1983 | - | | 0.8553 | 650 | 0.2101 | - | | 0.9211 | 700 | 0.2887 | - | | 0.9868 | 750 | 0.1288 | - | | 1.0 | 760 | - | 0.1818 | | 1.0526 | 800 | 0.135 | - | | 1.1184 | 850 | 0.1926 | - | | 1.1842 | 900 | 0.2656 | - | | 1.25 | 950 | 0.2925 | - | | 1.3158 | 1000 | 0.1753 | - | | 1.3816 | 1050 | 0.1736 | - | | 1.4474 | 1100 | 0.1193 | - | | 1.5132 | 1150 | 0.3204 | - | | 1.5789 | 1200 | 0.1234 | - | | 1.6447 | 1250 | 0.2137 | - | | 1.7105 | 1300 | 0.2369 | - | | 1.7763 | 1350 | 0.0442 | - | | 1.8421 | 1400 | 0.2339 | - | | 1.9079 | 1450 | 0.0696 | - | | 1.9737 | 1500 | 0.165 | - | | 2.0 | 1520 | - | 0.1262 | | 2.0395 | 1550 | 0.2138 | - | | 2.1053 | 1600 | 0.2716 | - | | 2.1711 | 1650 | 0.2227 | - | | 2.2368 | 1700 | 0.0874 | - | | 2.3026 | 1750 | 0.1628 | - | | 2.3684 | 1800 | 0.1013 | - | | 2.4342 | 1850 | 0.2291 | - | | 2.5 | 1900 | 0.1265 | - | | 2.5658 | 1950 | 0.2164 | - | | 2.6316 | 2000 | 0.1013 | - | | 2.6974 | 2050 | 0.2875 | - | | 2.7632 | 2100 | 0.0874 | - | | 2.8289 | 2150 | 0.2339 | - | | 2.8947 | 2200 | 0.1161 | - | | 2.9605 | 2250 | 0.2916 | - | | 3.0 | 2280 | - | 0.0963 | | 3.0263 | 2300 | 0.2737 | - | | 3.0921 | 2350 | 0.024 | - | | 3.1579 | 2400 | 0.0918 | - | | 3.2237 | 2450 | 0.0954 | - | | 3.2895 | 2500 | 0.1423 | - | | 3.3553 | 2550 | 0.2102 | - | | 3.4211 | 2600 | 0.1804 | - | | 3.4868 | 2650 | 0.0382 | - | | 3.5526 | 2700 | 0.0969 | - | | 3.6184 | 2750 | 0.1773 | - | | 3.6842 | 2800 | 0.0258 | - | | 3.75 | 2850 | 0.0374 | - | | 3.8158 | 2900 | 0.1698 | - | | 3.8816 | 2950 | 0.2979 | - | | 3.9474 | 3000 | 0.2007 | - | | 4.0 | 3040 | - | 0.0588 | | 4.0132 | 3050 | 0.1153 | - | | 4.0789 | 3100 | 0.0844 | - | | 4.1447 | 3150 | 0.031 | - | | 4.2105 | 3200 | 0.0028 | - | | 4.2763 | 3250 | 0.1175 | - | | 4.3421 | 3300 | 0.0022 | - | | 4.4079 | 3350 | 0.0285 | - | | 4.4737 | 3400 | 0.0133 | - | | 4.5395 | 3450 | 0.0059 | - | | 4.6053 | 3500 | 0.1918 | - | | 4.6711 | 3550 | 0.231 | - | | 4.7368 | 3600 | 0.124 | - | | 4.8026 | 3650 | 0.1725 | - | | 4.8684 | 3700 | 0.1108 | - | | 4.9342 | 3750 | 0.0037 | - | | 5.0 | 3800 | 0.0066 | 0.0383 | | 5.0658 | 3850 | 0.1364 | - | | 5.1316 | 3900 | 0.0552 | - | | 5.1974 | 3950 | 0.0148 | - | | 5.2632 | 4000 | 0.197 | - | | 5.3289 | 4050 | 0.0061 | - | | 5.3947 | 4100 | 0.0028 | - | | 5.4605 | 4150 | 0.1852 | - | | 5.5263 | 4200 | 0.0048 | - | | 5.5921 | 4250 | 0.1187 | - | | 5.6579 | 4300 | 0.0017 | - | | 5.7237 | 4350 | 0.0998 | - | | 5.7895 | 4400 | 0.1208 | - | | 5.8553 | 4450 | 0.0898 | - | | 5.9211 | 4500 | 0.096 | - | | 5.9868 | 4550 | 0.0035 | - | | 6.0 | 4560 | - | 0.0329 | | 6.0526 | 4600 | 0.092 | - | | 6.1184 | 4650 | 0.1557 | - | | 6.1842 | 4700 | 0.1312 | - | | 6.25 | 4750 | 0.0021 | - | | 6.3158 | 4800 | 0.0013 | - | | 6.3816 | 4850 | 0.002 | - | | 6.4474 | 4900 | 0.1176 | - | | 6.5132 | 4950 | 0.0116 | - | | 6.5789 | 5000 | 0.0017 | - | | 6.6447 | 5050 | 0.0004 | - | | 6.7105 | 5100 | 0.0007 | - | | 6.7763 | 5150 | 0.0008 | - | | 6.8421 | 5200 | 0.0014 | - | | 6.9079 | 5250 | 0.0404 | - | | 6.9737 | 5300 | 0.0047 | - | | 7.0 | 5320 | - | 0.0258 | | 7.0395 | 5350 | 0.0187 | - | | 7.1053 | 5400 | 0.0651 | - | | 7.1711 | 5450 | 0.0113 | - | | 7.2368 | 5500 | 0.0012 | - | | 7.3026 | 5550 | 0.0009 | - | | 7.3684 | 5600 | 0.0021 | - | | 7.4342 | 5650 | 0.1142 | - | | 7.5 | 5700 | 0.0006 | - | | 7.5658 | 5750 | 0.0011 | - | | 7.6316 | 5800 | 0.0003 | - | | 7.6974 | 5850 | 0.0188 | - | | 7.7632 | 5900 | 0.0101 | - | | 7.8289 | 5950 | 0.0004 | - | | 7.8947 | 6000 | 0.0013 | - | | 7.9605 | 6050 | 0.0016 | - | | 8.0 | 6080 | - | 0.0203 | | 8.0263 | 6100 | 0.0013 | - | | 8.0921 | 6150 | 0.0028 | - | | 8.1579 | 6200 | 0.0005 | - | | 8.2237 | 6250 | 0.0155 | - | | 8.2895 | 6300 | 0.0184 | - | | 8.3553 | 6350 | 0.0005 | - | | 8.4211 | 6400 | 0.0018 | - | | 8.4868 | 6450 | 0.0034 | - | | 8.5526 | 6500 | 0.0005 | - | | 8.6184 | 6550 | 0.0848 | - | | 8.6842 | 6600 | 0.0004 | - | | 8.75 | 6650 | 0.0696 | - | | 8.8158 | 6700 | 0.0353 | - | | 8.8816 | 6750 | 0.0057 | - | | 8.9474 | 6800 | 0.0008 | - | | **9.0** | **6840** | **-** | **0.0183** | | 9.0132 | 6850 | 0.0182 | - | | 9.0789 | 6900 | 0.0053 | - | | 9.1447 | 6950 | 0.0006 | - | | 9.2105 | 7000 | 0.0025 | - | | 9.2763 | 7050 | 0.003 | - | | 9.3421 | 7100 | 0.0004 | - | | 9.4079 | 7150 | 0.1523 | - | | 9.4737 | 7200 | 0.0005 | - | | 9.5395 | 7250 | 0.0729 | - | | 9.6053 | 7300 | 0.0146 | - | | 9.6711 | 7350 | 0.0009 | - | | 9.7368 | 7400 | 0.0011 | - | | 9.8026 | 7450 | 0.0614 | - | | 9.8684 | 7500 | 0.0006 | - | | 9.9342 | 7550 | 0.0005 | - | | 10.0 | 7600 | 0.0003 | 0.0196 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.1 - Sentence Transformers: 2.2.2 - Transformers: 4.35.2 - PyTorch: 2.1.0+cu121 - Datasets: 2.16.0 - Tokenizers: 0.15.0 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```