--- base_model: sentence-transformers/all-MiniLM-L6-v2 library_name: sentence-transformers pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:59616 - loss:CosineSimilarityLoss widget: - source_sentence: We all need to do our part in slowing the spread of COVID-19 and practice social distancing. As a result, we'll be closing our retail store at 401 Richmond until the end of March. You can still shop online at https://t.co/D5X7rxW0CS. Take care and stay safe everyone. sentences: - 'Retailers step up to fight coronavirus | Chain Store Age #Retail #Walmart #Pittsburgh #coronavirus #coronapocolypse https://t.co/trV5Gzu2Gk' - '@TODAYshow Consumer reporter Vicky Ngyuen IS NOT qualified to answer medical questions, from Facebook or anywhere, regarding the #coronavirus #JustSayin' - please help us stock some food coz we are in danger of this near by crisis of covid 19 which is now in our neighbouring country Kenya though we are just requesting to anyone who can manage to help us with anything small coz we want to stock food at the orphanage - source_sentence: "Joining hands with the UAE's efforts to contain the spread of\ \ the coronavirus (COVID-19) and considering the well-being of our mall visitors\ \ and community, gym, play areas & amusement centres will stay closed.\r\r\ \n\r\r\nLuLu Hypermarket and other stores are operating in routine. https://t.co/SVOTBu6ZNu" sentences: - A supermarket in La Habra is trying to help local seniors during the COVID-19 pandemic by opening its doors a half-hour early each day exclusively for shoppers 65 and older.? https://t.co/U3An6bnU5L - "Response to COVID-19 (Novel Coronavirus)\r\r\n\r\r\nPost Winery\x92s retail store,\ \ including The Trellis Room, will be CLOSED to the public Monday, March 16. Further\ \ closure or limited hours will be announced the following week, Mon.Mar.23rd.\ \ POST Wine available at local stores. Be well? https://t.co/e9WSaIP3mf" - "Look where somebody #Parked at #Safeway here at #Oakland \r\r\n#ParkingLot #lol\ \ #coronavirus #stockup #apocalypse2020 https://t.co/oorfCF2I7T" - source_sentence: 'You know it''s the end of the world when @BootsUK Surgical Spirit is out of stock online and (won''t be receiving any further stock) - How am I supposed to disinfect my hands now ? #coronavirus #coronapocolypse #panic #shopping https://t.co/iQIpUz50Sc' sentences: - "I\x92ve been online shopping for the past 2 days now #coronavirus" - "Mum actually mounted a dispenser and soap. You wash your hand before you enter\ \ her supermarket. \r\r\n\r\r\nNo chance for Covid-19" - Australia will NOT run out of food I get that people are fearful but panic buying creates more anxiety including for others who can t get what they actually need Please be kind to each other including shop staff If we work together the outcome is much better 19 - source_sentence: 'To help you eat well and limit your movement during the #coronavirus crisis, you can ?download #applications for smartphones that make it easier to recover food at any time:?Find a restaurant or grocery store near you that can deliver healthy meals to your home.' sentences: - "Was at the supermarket today. Didn't buy toilet paper. #Rebel\r\r\n\r\r\n#toiletpapercrisis\ \ #covid_19 https://t.co/eVXkQLIdAZ" - 'Are we getting quarantined an shoukd I stock up on food #coronavirus #Covid19Walkout #housearrest' - "The opportunities & challenges caused by #reverselogistics are even more\ \ apparent with the current #COVID-19 pandemic. As many stores have closed their\ \ physical locations, even more consumers are turning to online shopping. https://t.co/jmewQM90Z7\r\ \r\n#ECommerce #Returns #Coronavirus https://t.co/CCVKSzz3OH" - source_sentence: Governor Cuomo Please order retail businesses to close My husband has to go to work and expose himself to the virus Store owners will not close unless you tell them they have to I have two friends fighting for their lives PLEASE CLOSE STORES NOW sentences: - "Went to the store today. Most canned food is gone. We needed toilet paper and\ \ actually got a pack of 12 rolls. We got milk and cat treats as well. Thank god\ \ for Save on Foods for still having some stuff. \r\r\nOver 100 cases of Covid-19\ \ in my province so panic buying is common." - "Clothier @LandsEnd has closed all of its retail stores through March 29th due\ \ to coronavirus pandemic. They will pay all of their employees during the time\ \ period.\r\r\n\r\r\nThis is their store at the Hunt Valley Towne Center. #HuntValley\ \ #coronavirus https://t.co/DIwNzAOLeb" - 'COVID-19: No Need For Panic Buying, Food Available At All Times - PM Assures https://t.co/K2ECAO51yA' --- # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Finetuning The model was finetuned on [Coronavirus tweets NLP - Text Classification](https://www.kaggle.com/datasets/datatattle/covid-19-nlp-text-classification?select=Corona_NLP_train.csv). The objective was agrupate tweets by sentiment, for this, the dataset was divided into pairs of tweets and the similarity between them defined by the sentiment label. The following labels were used: - Extremely Positive and Extremely Negative: 0.0 - Extremely Positive and Neutral: 0.2 - Extremely Positive and Positive: 0.7 - Extremely Positive and Negative: 0.2 - Extremely Negative and Neutral: 0.2 - Extremely Negative and Positive: 0.2 - Extremely Negative and Negative: 0.7 - Neutral and Positive: 0.5 - Neutral and Negative: 0.5 ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) - **Maximum Sequence Length:** 256 tokens - **Output Dimensionality:** 384 tokens - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("gianvr/sbert-tunned-covid") # Run inference sentences = [ 'Governor Cuomo Please order retail businesses to close My husband has to go to work and expose himself to the virus Store owners will not close unless you tell them they have to I have two friends fighting for their lives PLEASE CLOSE STORES NOW', 'Clothier @LandsEnd has closed all of its retail stores through March 29th due to coronavirus pandemic. They will pay all of their employees during the time period.\r\r\n\r\r\nThis is their store at the Hunt Valley Towne Center. #HuntValley #coronavirus https://t.co/DIwNzAOLeb', 'Went to the store today. Most canned food is gone. We needed toilet paper and actually got a pack of 12 rolls. We got milk and cat treats as well. Thank god for Save on Foods for still having some stuff. \r\r\nOver 100 cases of Covid-19 in my province so panic buying is common.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Dataset #### Coronavirus tweets NLP - Text Classification * Size: 59,616 training samples * Columns: sentence1, sentence2, and label * Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | label | |:--------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | | | | * Samples: | sentence1 | sentence2 | label | |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | BBC Business correspondent just now: "We're going to see a sustained demand for food" Pretty sure that's not news?#CoronaCrisis #covid19uk | A warning from Nick Talley, University of Newcastle professor and neuro-gastroenterologist, for those who need to hear it: https://t.co/UzRRswUSTS | 0.7 | | The announcement(over the PA)at #Woolworths #Marrickville Metro supermarket telling shoppers to maintain #distancing is absurd. In a supermarket aisle there just isn't enough room! #Auspol #NSWpol #coronavirus #Covid_19australia #COVID19Aus | Due to the closure of indoor shopping malls to prevent the spread of our Brooklyn Atlantic Office will be closed until further notice We apologize for the inconvenience Please visit our website to find more than 60 online transactions | 0.5 | | Expert s top tips for supermarket shopping 1 Avoid cash 2 Use a key to type in credit card pin 3 Don t take a receipt 4 Use hand sanitizer once back at your car 5 Consider quarantining food Read more | Covid-19 is likely to disrupt these data in the coming months. So far, there is only limited evidence of surging demand in supermarkets stoking prices (mostly non-food and bread). Meanwhile, fruit, veg, and sugary item prices have all crashed. 2/3 | 0.2 |0.7 | * Loss: [CosineSimilarityLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters: ```json { "loss_fct": "torch.nn.modules.loss.MSELoss" } ``` ### Training Hyperparameters #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: no - `prediction_loss_only`: True - `per_device_train_batch_size`: 8 - `per_device_eval_batch_size`: 8 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 3.0 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `eval_use_gather_object`: False - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | |:------:|:-----:|:-------------:| | 0.0671 | 500 | 0.1053 | | 0.1342 | 1000 | 0.0802 | | 0.2013 | 1500 | 0.0741 | | 0.2684 | 2000 | 0.0701 | | 0.3355 | 2500 | 0.0682 | | 0.4026 | 3000 | 0.0665 | | 0.4697 | 3500 | 0.0674 | | 0.5368 | 4000 | 0.0667 | | 0.6039 | 4500 | 0.0624 | | 0.6710 | 5000 | 0.0613 | | 0.7381 | 5500 | 0.0589 | | 0.8052 | 6000 | 0.0647 | | 0.8722 | 6500 | 0.0645 | | 0.9393 | 7000 | 0.0626 | | 1.0064 | 7500 | 0.0616 | | 1.0735 | 8000 | 0.0591 | | 1.1406 | 8500 | 0.0601 | | 1.2077 | 9000 | 0.0592 | | 1.2748 | 9500 | 0.0562 | | 1.3419 | 10000 | 0.0586 | | 1.4090 | 10500 | 0.0593 | | 1.4761 | 11000 | 0.0578 | | 1.5432 | 11500 | 0.0574 | | 1.6103 | 12000 | 0.0561 | | 1.6774 | 12500 | 0.0579 | | 1.7445 | 13000 | 0.0556 | | 1.8116 | 13500 | 0.0551 | | 1.8787 | 14000 | 0.0554 | | 1.9458 | 14500 | 0.0537 | | 2.0129 | 15000 | 0.0548 | | 2.0800 | 15500 | 0.0521 | | 2.1471 | 16000 | 0.0534 | | 2.2142 | 16500 | 0.0529 | | 2.2813 | 17000 | 0.0529 | | 2.3484 | 17500 | 0.052 | | 2.4155 | 18000 | 0.0527 | | 2.4826 | 18500 | 0.0517 | | 2.5497 | 19000 | 0.0517 | | 2.6167 | 19500 | 0.0497 | | 2.6838 | 20000 | 0.0501 | | 2.7509 | 20500 | 0.0516 | | 2.8180 | 21000 | 0.0515 | | 2.8851 | 21500 | 0.0511 | | 2.9522 | 22000 | 0.0489 | ### Framework Versions - Python: 3.10.12 - Sentence Transformers: 3.2.1 - Transformers: 4.44.2 - PyTorch: 2.5.0+cu121 - Accelerate: 0.34.2 - Datasets: 3.1.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```