--- library_name: setfit metrics: - accuracy pipeline_tag: text-classification tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: 'Se propone la práctica de los siguientes medios de prueba: análisis del conductor habitual del vehículo y testimonio del agente denunciante, para demostrar la inexistencia de infracción.' - text: Deseo aclarar que mi automóvil no obstaculizaba en absoluto el tráfico, ya que estaba correctamente aparcado y dejaba espacio suficiente para la circulación de otros vehículos. - text: No quiero ser sancionado dos veces por el mismo hecho, es una violación del principio de non bis in idem - text: Que teniendo por presentado este escrito, se sirva admitirlo, teniendo por formuladas las alegaciones en el mismo contenidas y, en consecuencia, curse las oportunas instrucciones con el fín de que se proceda a la anulación del expediente sancionador referido - text: Me ha llegado una denuncia de tráfico del coche 1234HYG, pero este automóvil fue dado de baja por robo en mayo de 2023 y aún no ha sido recuperado. Pido que se anule la sanción ya que el vehículo no podía estar circulando en la fecha de la infracción. inference: true model-index: - name: SetFit results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.879746835443038 name: Accuracy --- # SetFit This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 20 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 353 | | | 78 | | | 2001 | | | 2017 | | | 304 | | | 49 | | | 1001 | | | 2039 | | | 237 | | | 2014 | | | 2060 | | | 2026 | | | 2038 | | | 2013 | | | 994 | | | 2002 | | | 357 | | | 2037 | | | 2010 | | | 2022 | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.8797 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("desarrolloasesoreslocales/bert-leg-al-setfit") # Run inference preds = model("No quiero ser sancionado dos veces por el mismo hecho, es una violación del principio de non bis in idem") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 3 | 34.3883 | 213 | | Label | Training Sample Count | |:------|:----------------------| | 49 | 32 | | 78 | 32 | | 237 | 32 | | 304 | 32 | | 353 | 32 | | 357 | 32 | | 994 | 32 | | 1001 | 32 | | 2001 | 32 | | 2002 | 23 | | 2010 | 32 | | 2013 | 32 | | 2014 | 32 | | 2017 | 32 | | 2022 | 32 | | 2026 | 32 | | 2037 | 32 | | 2038 | 32 | | 2039 | 32 | | 2060 | 32 | ### Training Hyperparameters - batch_size: (32, 32) - num_epochs: (1, 1) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 10 - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 1e-06 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: True - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:----------:|:------:|:-------------:|:---------------:| | 0.0025 | 1 | 0.0002 | - | | **0.1266** | **50** | **0.0001** | **0.0673** | | 0.2532 | 100 | 0.0001 | 0.0701 | | 0.3797 | 150 | 0.0005 | 0.0723 | | 0.5063 | 200 | 0.0001 | 0.0698 | | 0.6329 | 250 | 0.0012 | 0.0731 | | 0.7595 | 300 | 0.0001 | 0.0728 | | 0.8861 | 350 | 0.0 | 0.0745 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.3 - Sentence Transformers: 3.0.1 - Transformers: 4.40.2 - PyTorch: 2.3.1+cu121 - Datasets: 2.20.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```