Top2Vec Scientific Texts Model
This repository hosts the top2vec_scientific_texts model, a specialized Top2Vec model trained on scientific texts for topic modeling and semantic search.
Model Overview
The top2vec_scientific_texts model is built for analyzing scientific literature. It leverages the Universal Sentence Encoder for embedding texts and uses Top2Vec for topic modeling.
Key Features:
- Domain-Specific: Tailored for scientific texts.
- Base Model: Utilizes the Universal Sentence Encoder for effective text embeddings.
- Topic Modeling: Employs Top2Vec for discovering topics in scientific documents.
Installation
To use the model, you need to install the following dependencies:
pip install top2vec
pip install top2vec[sentence_encoders]
pip install tensorflow==2.8.0
pip install tensorflow-probability==0.16.0
Model Training Process
The entire process of model training, dataset creation, and visualization is documented in the main.ipynb Jupyter notebook. To explore the code and replicate the results:
- Open the
main.ipynbnotebook in Jupyter Lab or Jupyter Notebook. - Execute the cells in sequence to run different stages of the analysis.
- The results, including thematic group analysis, trend analysis, and visualizations of interest dynamics over the years, are presented in the form of tables and graphs within the notebook.
For more details, please refer to the main.ipynb notebook in this repository.
Usage
Here's an example of how to use the model for topic modeling:
from top2vec import Top2Vec
# Load your documents
docs = ["Document 1 text", "Document 2 text", ...]
# Initialize the Top2Vec model
model = Top2Vec(
documents=docs,
speed='learn',
workers=80,
embedding_model='universal-sentence-encoder',
umap_args={'n_neighbors': 15, 'n_components': 5, 'metric': 'cosine', 'min_dist': 0.0, 'random_state': 42},
hdbscan_args={'min_cluster_size': 15, 'metric': 'euclidean', 'cluster_selection_method': 'eom'}
)
Save the model
model.save('top2vec_scientific_texts_model')
Dataset
The model was trained on a dataset of scientific abstracts sourced from arXiv. The dataset covers a range of topics within the field of computer science from 2010 to 2024.
You can access the dataset arxiv_papers_cs.
Use Cases
The top2vec_scientific_texts model can be used for various purposes, including:
- Topic Discovery: Identify the main topics within a collection of scientific texts.
- Semantic Search: Find documents that are semantically similar to a query text.
- Trend Analysis: Analyze the evolution of topics over time.
Examples
Here are some examples of the model's output for the thematic group "UAV in Disasters and Emergency":
Trend Analysis for "UAV in Disasters and Emergency"
This graph shows the trend of interest in the use of UAVs in disaster and emergency situations over time.
Key Metrics Table
Analysis for Thematic Group: Disasters & Emergency
| Year | Number of Publications | Growth Acceleration | Change in Number of Publications | Relative Growth |
|---|---|---|---|---|
| 2010 | 19 | 0 | 0 | 0.0% |
| 2011 | 15 | -4 | -4 | -21.05% |
| 2012 | 28 | 17 | 13 | 86.67% |
| 2013 | 38 | -3 | 10 | 35.71% |
| 2014 | 28 | -20 | -10 | -26.32% |
| 2015 | 47 | 29 | 19 | 67.86% |
| 2016 | 63 | -3 | 16 | 34.04% |
| 2017 | 94 | 15 | 31 | 49.21% |
| 2018 | 173 | 48 | 79 | 84.04% |
| 2019 | 266 | 14 | 93 | 53.76% |
| 2020 | 337 | -22 | 71 | 26.69% |
| 2021 | 380 | -28 | 43 | 12.76% |
| 2022 | 453 | 30 | 73 | 19.21% |
| 2023 | 509 | -17 | 56 | 12.36% |
Contributions
We welcome contributions to the top2vec_scientific_texts model. If you have suggestions, improvements, or encounter any issues, please feel free to open an issue or submit a pull request.
License
This project is licensed under the MIT License