--- library_name: setfit tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer metrics: - accuracy widget: - text: MOVED Follow mhonkasalo for updates Ethereum - text: 'Buy online with Bitcoin Dash and other cryptocurrencies ' - text: 'Blockchain cryptographer Research partner a16z crypto ' - text: 'A Tezos News Journalistic Hub Tweets or website content are NOT investment or financial advice Tweets Or Retweets are not endorsements Tezos ' - text: Ethereans en el construyendo el futuro de la Web3 Te esperamos el 3 de febrero en el Ethereum Lima Day Peru pipeline_tag: text-classification inference: true base_model: BAAI/bge-small-en-v1.5 model-index: - name: SetFit with BAAI/bge-small-en-v1.5 results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.4918032786885246 name: Accuracy --- # SetFit with BAAI/bge-small-en-v1.5 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 50 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | DEFI | | | WALLET | | | INFRASTRUCTURE | | | LSD | | | CENTRALIZED_EXCHANGE | | | NFT | | | DEVELOPMENT_AGENCY | | | PRIVACY | | | PAYMENT_PROVIDER | | | UNDETERMINED | | | DAO | | | CRYPTO_MEDIA | | | METAVERSE | | | SUPPLY_CHAIN | | | VENTURE_CAPITAL_FIRM | | | RESEARCH_AGENCY | | | MARKETING_AGENCY | | | LEGAL_COMPLIANCE | | | L1_BLOCKCHAIN | | | REFI | | | GAMEFI | | | NFT_MARKETPLACE | | | L0_BLOCKCHAIN | | | L2_BLOCKCHAIN | | | SOCIAL_MEDIA | | | CHARITY | | | NFT_DIGITAL_ART | | | NFT_GAMING | | | FOUNDATION | | | REAL_ESTATE | | | DECENTRALIZED_STORAGE | | | DEX | | | LENDING_BORROWING | | | SOCIALFI | | | PODCAST | | | MEME_COIN | | | SYNTHETIC_ASSETS | | | YIELD_FARMING | | | GOVERNMENT | | | PERPS | | | DECENTRALIZED_COMPUTING | | | STABLECOIN | | | NFT_IDENTITY | | | INSURANCE | | | RWA | | | GAMBLEFI | | | L3_BLOCKCHAIN | | | OPTIONS | | | OTC_EXCHANGE | | | HEALTHCARE | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.4918 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("kasparas12/crypto_organization_infer_model_setfit") # Run inference preds = model("MOVED Follow mhonkasalo for updates Ethereum") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 2 | 16.1218 | 45 | | Label | Training Sample Count | |:------------------------|:----------------------| | DEVELOPMENT_AGENCY | 308 | | RESEARCH_AGENCY | 367 | | MARKETING_AGENCY | 107 | | FOUNDATION | 128 | | CHARITY | 72 | | L0_BLOCKCHAIN | 27 | | L1_BLOCKCHAIN | 170 | | L2_BLOCKCHAIN | 143 | | L3_BLOCKCHAIN | 3 | | VENTURE_CAPITAL_FIRM | 550 | | GOVERNMENT | 54 | | CENTRALIZED_EXCHANGE | 124 | | OTC_EXCHANGE | 3 | | DEX | 162 | | LENDING_BORROWING | 36 | | INSURANCE | 14 | | YIELD_FARMING | 22 | | SYNTHETIC_ASSETS | 9 | | LSD | 54 | | PERPS | 11 | | OPTIONS | 18 | | WALLET | 171 | | STABLECOIN | 33 | | DEFI | 770 | | NFT | 121 | | NFT_MARKETPLACE | 100 | | NFT_DIGITAL_ART | 278 | | NFT_GAMING | 181 | | NFT_IDENTITY | 64 | | PRIVACY | 97 | | DECENTRALIZED_STORAGE | 110 | | DECENTRALIZED_COMPUTING | 38 | | SOCIALFI | 54 | | SOCIAL_MEDIA | 54 | | SUPPLY_CHAIN | 9 | | REAL_ESTATE | 12 | | REFI | 49 | | HEALTHCARE | 8 | | LEGAL_COMPLIANCE | 92 | | GAMEFI | 20 | | GAMBLEFI | 18 | | INFRASTRUCTURE | 649 | | RWA | 19 | | METAVERSE | 59 | | MEME_COIN | 37 | | PAYMENT_PROVIDER | 89 | | DAO | 522 | | CRYPTO_MEDIA | 829 | | PODCAST | 89 | | UNDETERMINED | 728 | ### Training Hyperparameters - batch_size: (64, 64) - num_epochs: (1, 1) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 20 - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: False ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:------:|:----:|:-------------:|:---------------:| | 0.0002 | 1 | 0.232 | - | | 0.0104 | 50 | 0.2604 | - | | 0.0208 | 100 | 0.2484 | - | | 0.0312 | 150 | 0.2445 | - | | 0.0416 | 200 | 0.2294 | - | | 0.0521 | 250 | 0.2207 | - | | 0.0625 | 300 | 0.1996 | - | | 0.0729 | 350 | 0.2222 | - | | 0.0833 | 400 | 0.206 | - | | 0.0937 | 450 | 0.1937 | - | | 0.1041 | 500 | 0.1651 | - | | 0.1145 | 550 | 0.2341 | - | | 0.1249 | 600 | 0.1862 | - | | 0.1354 | 650 | 0.1922 | - | | 0.1458 | 700 | 0.1987 | - | | 0.1562 | 750 | 0.1537 | - | | 0.1666 | 800 | 0.1281 | - | | 0.1770 | 850 | 0.102 | - | | 0.1874 | 900 | 0.1395 | - | | 0.1978 | 950 | 0.1816 | - | | 0.2082 | 1000 | 0.1109 | - | | 0.2187 | 1050 | 0.0924 | - | | 0.2291 | 1100 | 0.089 | - | | 0.2395 | 1150 | 0.1228 | - | | 0.2499 | 1200 | 0.1303 | - | | 0.2603 | 1250 | 0.1084 | - | | 0.2707 | 1300 | 0.1483 | - | | 0.2811 | 1350 | 0.1545 | - | | 0.2915 | 1400 | 0.129 | - | | 0.3020 | 1450 | 0.1177 | - | | 0.3124 | 1500 | 0.1936 | - | | 0.3228 | 1550 | 0.1427 | - | | 0.3332 | 1600 | 0.0968 | - | | 0.3436 | 1650 | 0.1252 | - | | 0.3540 | 1700 | 0.0896 | - | | 0.3644 | 1750 | 0.1281 | - | | 0.3748 | 1800 | 0.0965 | - | | 0.3853 | 1850 | 0.0725 | - | | 0.3957 | 1900 | 0.0625 | - | | 0.4061 | 1950 | 0.1 | - | | 0.4165 | 2000 | 0.086 | - | | 0.4269 | 2050 | 0.0793 | - | | 0.4373 | 2100 | 0.1193 | - | | 0.4477 | 2150 | 0.0812 | - | | 0.4581 | 2200 | 0.1102 | - | | 0.4686 | 2250 | 0.0862 | - | | 0.4790 | 2300 | 0.0749 | - | | 0.4894 | 2350 | 0.0864 | - | | 0.4998 | 2400 | 0.0974 | - | | 0.5102 | 2450 | 0.0707 | - | | 0.5206 | 2500 | 0.0981 | - | | 0.5310 | 2550 | 0.098 | - | | 0.5414 | 2600 | 0.0787 | - | | 0.5519 | 2650 | 0.1141 | - | | 0.5623 | 2700 | 0.0705 | - | | 0.5727 | 2750 | 0.0922 | - | | 0.5831 | 2800 | 0.0713 | - | | 0.5935 | 2850 | 0.1087 | - | | 0.6039 | 2900 | 0.0311 | - | | 0.6143 | 2950 | 0.0674 | - | | 0.6247 | 3000 | 0.0472 | - | | 0.6352 | 3050 | 0.0591 | - | | 0.6456 | 3100 | 0.0641 | - | | 0.6560 | 3150 | 0.0925 | - | | 0.6664 | 3200 | 0.0473 | - | | 0.6768 | 3250 | 0.0461 | - | | 0.6872 | 3300 | 0.0778 | - | | 0.6976 | 3350 | 0.0818 | - | | 0.7080 | 3400 | 0.0603 | - | | 0.7185 | 3450 | 0.0603 | - | | 0.7289 | 3500 | 0.0633 | - | | 0.7393 | 3550 | 0.09 | - | | 0.7497 | 3600 | 0.079 | - | | 0.7601 | 3650 | 0.0814 | - | | 0.7705 | 3700 | 0.0433 | - | | 0.7809 | 3750 | 0.0425 | - | | 0.7913 | 3800 | 0.0858 | - | | 0.8017 | 3850 | 0.0601 | - | | 0.8122 | 3900 | 0.052 | - | | 0.8226 | 3950 | 0.1124 | - | | 0.8330 | 4000 | 0.0666 | - | | 0.8434 | 4050 | 0.0744 | - | | 0.8538 | 4100 | 0.099 | - | | 0.8642 | 4150 | 0.0734 | - | | 0.8746 | 4200 | 0.0996 | - | | 0.8850 | 4250 | 0.0761 | - | | 0.8955 | 4300 | 0.0848 | - | | 0.9059 | 4350 | 0.0414 | - | | 0.9163 | 4400 | 0.0596 | - | | 0.9267 | 4450 | 0.067 | - | | 0.9371 | 4500 | 0.1015 | - | | 0.9475 | 4550 | 0.0602 | - | | 0.9579 | 4600 | 0.0496 | - | | 0.9683 | 4650 | 0.053 | - | | 0.9788 | 4700 | 0.0922 | - | | 0.9892 | 4750 | 0.0853 | - | | 0.9996 | 4800 | 0.0912 | - | ### Framework Versions - Python: 3.9.16 - SetFit: 1.0.3 - Sentence Transformers: 2.2.2 - Transformers: 4.21.3 - PyTorch: 1.12.1+cu116 - Datasets: 2.4.0 - Tokenizers: 0.12.1 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```