Add BERTopic model

README.md

@@ -0,0 +1,174 @@
+
+---
+tags:
+- bertopic
+library_name: bertopic
+pipeline_tag: text-classification
+---
+
+# BERTopic_ArXiv
+
+This is a [BERTopic](https://github.com/MaartenGr/BERTopic) model. 
+BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets. 
+
+## Usage 
+
+To use this model, please install BERTopic:
+
+```
+pip install -U bertopic
+```
+
+You can use the model as follows:
+
+```python
+from bertopic import BERTopic
+topic_model = BERTopic.load("MaartenGr/BERTopic_ArXiv")
+
+topic_model.get_topic_info()
+```
+
+## Topic overview
+
+* Number of topics: 107
+* Number of training documents: 33189
+
+<details>
+  <summary>Click here for an overview of all topics.</summary>
+  
+  | Topic ID | Topic Keywords | Topic Frequency | Label | 
+|----------|----------------|-----------------|-------| 
+| -1 | language - models - model - data - based | 20 | -1_language_models_model_data | 
+| 0 | dialogue - dialog - response - responses - intent | 14247 | 0_dialogue_dialog_response_responses | 
+| 1 | speech - asr - speech recognition - recognition - end | 1833 | 1_speech_asr_speech recognition_recognition | 
+| 2 | tuning - tasks - prompt - models - language | 1369 | 2_tuning_tasks_prompt_models | 
+| 3 | summarization - summaries - summary - abstractive - document | 1109 | 3_summarization_summaries_summary_abstractive | 
+| 4 | question - answer - qa - answering - question answering | 893 | 4_question_answer_qa_answering | 
+| 5 | sentiment - sentiment analysis - aspect - analysis - opinion | 837 | 5_sentiment_sentiment analysis_aspect_analysis | 
+| 6 | clinical - medical - biomedical - notes - patient | 691 | 6_clinical_medical_biomedical_notes | 
+| 7 | translation - nmt - machine translation - neural machine - neural machine translation | 586 | 7_translation_nmt_machine translation_neural machine | 
+| 8 | generation - text generation - text - language generation - nlg | 558 | 8_generation_text generation_text_language generation | 
+| 9 | hate - hate speech - offensive - speech - detection | 484 | 9_hate_hate speech_offensive_speech | 
+| 10 | news - fake - fake news - stance - fact | 455 | 10_news_fake_fake news_stance | 
+| 11 | relation - relation extraction - extraction - relations - entity | 450 | 11_relation_relation extraction_extraction_relations | 
+| 12 | ner - named - named entity - entity - named entity recognition | 376 | 12_ner_named_named entity_entity | 
+| 13 | parsing - parser - dependency - treebank - parsers | 370 | 13_parsing_parser_dependency_treebank | 
+| 14 | event - temporal - events - event extraction - extraction | 314 | 14_event_temporal_events_event extraction | 
+| 15 | emotion - emotions - multimodal - emotion recognition - emotional | 300 | 15_emotion_emotions_multimodal_emotion recognition | 
+| 16 | word - embeddings - word embeddings - embedding - words | 292 | 16_word_embeddings_word embeddings_embedding | 
+| 17 | explanations - explanation - rationales - rationale - interpretability | 212 | 17_explanations_explanation_rationales_rationale | 
+| 18 | morphological - arabic - morphology - languages - inflection | 204 | 18_morphological_arabic_morphology_languages | 
+| 19 | topic - topics - topic models - lda - topic modeling | 200 | 19_topic_topics_topic models_lda | 
+| 20 | bias - gender - biases - gender bias - debiasing | 195 | 20_bias_gender_biases_gender bias | 
+| 21 | law - frequency - zipf - words - length | 185 | 21_law_frequency_zipf_words | 
+| 22 | legal - court - law - legal domain - case | 182 | 22_legal_court_law_legal domain | 
+| 23 | adversarial - attacks - attack - adversarial examples - robustness | 181 | 23_adversarial_attacks_attack_adversarial examples | 
+| 24 | commonsense - commonsense knowledge - reasoning - knowledge - commonsense reasoning | 180 | 24_commonsense_commonsense knowledge_reasoning_knowledge | 
+| 25 | quantum - semantics - calculus - compositional - meaning | 171 | 25_quantum_semantics_calculus_compositional | 
+| 26 | correction - error - error correction - grammatical - grammatical error | 161 | 26_correction_error_error correction_grammatical | 
+| 27 | argument - arguments - argumentation - argumentative - mining | 160 | 27_argument_arguments_argumentation_argumentative | 
+| 28 | sarcasm - humor - sarcastic - detection - humorous | 157 | 28_sarcasm_humor_sarcastic_detection | 
+| 29 | coreference - resolution - coreference resolution - mentions - mention | 156 | 29_coreference_resolution_coreference resolution_mentions | 
+| 30 | sense - word sense - wsd - word - disambiguation | 153 | 30_sense_word sense_wsd_word | 
+| 31 | knowledge - knowledge graph - graph - link prediction - entities | 149 | 31_knowledge_knowledge graph_graph_link prediction | 
+| 32 | parsing - semantic parsing - amr - semantic - parser | 146 | 32_parsing_semantic parsing_amr_semantic | 
+| 33 | cross lingual - lingual - cross - transfer - languages | 146 | 33_cross lingual_lingual_cross_transfer | 
+| 34 | mt - translation - qe - quality - machine translation | 139 | 34_mt_translation_qe_quality | 
+| 35 | sql - text sql - queries - spider - schema | 138 | 35_sql_text sql_queries_spider | 
+| 36 | classification - text classification - label - text - labels | 136 | 36_classification_text classification_label_text | 
+| 37 | style - style transfer - transfer - text style - text style transfer | 136 | 37_style_style transfer_transfer_text style | 
+| 38 | question - question generation - questions - answer - generation | 129 | 38_question_question generation_questions_answer | 
+| 39 | authorship - authorship attribution - attribution - author - authors | 127 | 39_authorship_authorship attribution_attribution_author | 
+| 40 | sentence - sentence embeddings - similarity - sts - sentence embedding | 123 | 40_sentence_sentence embeddings_similarity_sts | 
+| 41 | code - identification - switching - cs - code switching | 121 | 41_code_identification_switching_cs | 
+| 42 | story - stories - story generation - generation - storytelling | 118 | 42_story_stories_story generation_generation | 
+| 43 | discourse - discourse relation - discourse relations - rst - discourse parsing | 117 | 43_discourse_discourse relation_discourse relations_rst | 
+| 44 | code - programming - source code - code generation - programming languages | 117 | 44_code_programming_source code_code generation | 
+| 45 | paraphrase - paraphrases - paraphrase generation - paraphrasing - generation | 114 | 45_paraphrase_paraphrases_paraphrase generation_paraphrasing | 
+| 46 | agent - games - environment - instructions - agents | 111 | 46_agent_games_environment_instructions | 
+| 47 | covid - covid 19 - 19 - tweets - pandemic | 108 | 47_covid_covid 19_19_tweets | 
+| 48 | linking - entity linking - entity - el - entities | 107 | 48_linking_entity linking_entity_el | 
+| 49 | poetry - poems - lyrics - poem - music | 103 | 49_poetry_poems_lyrics_poem | 
+| 50 | image - captioning - captions - visual - caption | 100 | 50_image_captioning_captions_visual | 
+| 51 | nli - entailment - inference - natural language inference - language inference | 96 | 51_nli_entailment_inference_natural language inference | 
+| 52 | keyphrase - keyphrases - extraction - document - phrases | 95 | 52_keyphrase_keyphrases_extraction_document | 
+| 53 | simplification - text simplification - ts - sentence - simplified | 95 | 53_simplification_text simplification_ts_sentence | 
+| 54 | empathetic - emotion - emotional - empathy - emotions | 95 | 54_empathetic_emotion_emotional_empathy | 
+| 55 | depression - mental - health - mental health - social media | 93 | 55_depression_mental_health_mental health | 
+| 56 | segmentation - word segmentation - chinese - chinese word segmentation - chinese word | 93 | 56_segmentation_word segmentation_chinese_chinese word segmentation | 
+| 57 | citation - scientific - papers - citations - scholarly | 85 | 57_citation_scientific_papers_citations | 
+| 58 | agreement - syntactic - verb - grammatical - subject verb | 85 | 58_agreement_syntactic_verb_grammatical | 
+| 59 | metaphor - literal - figurative - metaphors - idiomatic | 83 | 59_metaphor_literal_figurative_metaphors | 
+| 60 | srl - semantic role - role labeling - semantic role labeling - role | 82 | 60_srl_semantic role_role labeling_semantic role labeling | 
+| 61 | privacy - private - federated - privacy preserving - federated learning | 82 | 61_privacy_private_federated_privacy preserving | 
+| 62 | change - semantic change - time - semantic - lexical semantic | 82 | 62_change_semantic change_time_semantic | 
+| 63 | bilingual - lingual - cross lingual - cross - embeddings | 80 | 63_bilingual_lingual_cross lingual_cross | 
+| 64 | political - media - news - bias - articles | 77 | 64_political_media_news_bias | 
+| 65 | medical - qa - question - questions - clinical | 75 | 65_medical_qa_question_questions | 
+| 66 | math - mathematical - math word - word problems - problems | 73 | 66_math_mathematical_math word_word problems | 
+| 67 | financial - stock - market - price - news | 69 | 67_financial_stock_market_price | 
+| 68 | table - tables - tabular - reasoning - qa | 69 | 68_table_tables_tabular_reasoning | 
+| 69 | readability - complexity - assessment - features - reading | 65 | 69_readability_complexity_assessment_features | 
+| 70 | layout - document - documents - document understanding - extraction | 64 | 70_layout_document_documents_document understanding | 
+| 71 | brain - cognitive - reading - syntactic - language | 62 | 71_brain_cognitive_reading_syntactic | 
+| 72 | sign - gloss - language - signed - language translation | 61 | 72_sign_gloss_language_signed | 
+| 73 | vqa - visual - visual question - visual question answering - question | 59 | 73_vqa_visual_visual question_visual question answering | 
+| 74 | biased - biases - spurious - nlp - debiasing | 57 | 74_biased_biases_spurious_nlp | 
+| 75 | visual - dialogue - multimodal - image - dialog | 55 | 75_visual_dialogue_multimodal_image | 
+| 76 | translation - machine translation - machine - smt - statistical | 54 | 76_translation_machine translation_machine_smt | 
+| 77 | multimodal - visual - image - translation - machine translation | 52 | 77_multimodal_visual_image_translation | 
+| 78 | geographic - location - geolocation - geo - locations | 51 | 78_geographic_location_geolocation_geo | 
+| 79 | reasoning - prompting - llms - chain thought - chain | 48 | 79_reasoning_prompting_llms_chain thought | 
+| 80 | essay - scoring - aes - essay scoring - essays | 45 | 80_essay_scoring_aes_essay scoring | 
+| 81 | crisis - disaster - traffic - tweets - disasters | 45 | 81_crisis_disaster_traffic_tweets | 
+| 82 | graph - text classification - text - gcn - classification | 44 | 82_graph_text classification_text_gcn | 
+| 83 | annotation - tools - linguistic - resources - xml | 43 | 83_annotation_tools_linguistic_resources | 
+| 84 | entity alignment - alignment - kgs - entity - ea | 43 | 84_entity alignment_alignment_kgs_entity | 
+| 85 | personality - traits - personality traits - evaluative - text | 42 | 85_personality_traits_personality traits_evaluative | 
+| 86 | ad - alzheimer - alzheimer disease - disease - speech | 40 | 86_ad_alzheimer_alzheimer disease_disease | 
+| 87 | taxonomy - hypernymy - taxonomies - hypernym - hypernyms | 39 | 87_taxonomy_hypernymy_taxonomies_hypernym | 
+| 88 | active learning - active - al - learning - uncertainty | 37 | 88_active learning_active_al_learning | 
+| 89 | reviews - summaries - summarization - review - opinion | 36 | 89_reviews_summaries_summarization_review | 
+| 90 | emoji - emojis - sentiment - message - anonymous | 35 | 90_emoji_emojis_sentiment_message | 
+| 91 | table - table text - tables - table text generation - text generation | 35 | 91_table_table text_tables_table text generation | 
+| 92 | domain - domain adaptation - adaptation - domains - source | 35 | 92_domain_domain adaptation_adaptation_domains | 
+| 93 | alignment - word alignment - parallel - pairs - alignments | 34 | 93_alignment_word alignment_parallel_pairs | 
+| 94 | indo - languages - indo european - names - family | 34 | 94_indo_languages_indo european_names | 
+| 95 | patent - claim - claim generation - chemical - technical | 32 | 95_patent_claim_claim generation_chemical | 
+| 96 | agents - emergent - communication - referential - games | 32 | 96_agents_emergent_communication_referential | 
+| 97 | graph - amr - graph text - graphs - text generation | 31 | 97_graph_amr_graph text_graphs | 
+| 98 | moral - ethical - norms - values - social | 29 | 98_moral_ethical_norms_values | 
+| 99 | acronym - acronyms - abbreviations - abbreviation - disambiguation | 27 | 99_acronym_acronyms_abbreviations_abbreviation | 
+| 100 | typing - entity typing - entity - type - types | 27 | 100_typing_entity typing_entity_type | 
+| 101 | coherence - discourse - discourse coherence - coherence modeling - text | 26 | 101_coherence_discourse_discourse coherence_coherence modeling | 
+| 102 | pos - taggers - tagging - tagger - pos tagging | 25 | 102_pos_taggers_tagging_tagger | 
+| 103 | drug - social - social media - media - health | 25 | 103_drug_social_social media_media | 
+| 104 | gender - translation - bias - gender bias - mt | 24 | 104_gender_translation_bias_gender bias | 
+| 105 | job - resume - skills - skill - soft | 21 | 105_job_resume_skills_skill |
+  
+</details>
+
+## Training hyperparameters
+
+* calculate_probabilities: False
+* language: None
+* low_memory: False
+* min_topic_size: 10
+* n_gram_range: (1, 1)
+* nr_topics: None
+* seed_topic_list: None
+* top_n_words: 10
+* verbose: True
+
+## Framework versions
+
+* Numpy: 1.22.4
+* HDBSCAN: 0.8.29
+* UMAP: 0.5.3
+* Pandas: 1.5.3
+* Scikit-Learn: 1.2.2
+* Sentence-transformers: 2.2.2
+* Transformers: 4.29.2
+* Numba: 0.56.4
+* Plotly: 5.13.1
+* Python: 3.10.11

config.json

@@ -0,0 +1,15 @@
+{
+  "calculate_probabilities": false,
+  "language": null,
+  "low_memory": false,
+  "min_topic_size": 10,
+  "n_gram_range": [
+    1,
+    1
+  ],
+  "nr_topics": null,
+  "seed_topic_list": null,
+  "top_n_words": 10,
+  "verbose": true,
+  "embedding_model": "sentence-transformers/all-mpnet-base-v2"
+}

ctfidf.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:547360497eca2939ce70813b95f0234945004863f67c0e21ced47b26ca3e4dca
+size 9317840

topic_embeddings.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c96563fb1306ff6b742112a98e27f85ec911c27c11d3b381e1cb849a1a72088
+size 328792