Large language model (LLM) trends and taxon- omy have changed rapidly in the last few years, primarily due to the advancement of data sci- ences like natural language processing (NLP), deep learning, and the ever-growing size of computational resources. These models aim to enhance logical and mathematical reasoning be- yond pattern recognition. This work aims to ex- plore trends in survey papers over time and an- alyze their associated taxonomies through data exploration, visualization, and machine learn- ing modeling. Initially, the dataset of survey papers is preprocessed by grouping the number of surveys by year and month, revealing publi- cation trends across time. A detailed analysis of taxonomy distributions is performed to iden- tify the prevalence of various survey categories. Using the TF-IDF method, the titles and sum- maries of papers are vectorized, transforming textual information into numerical features. A one-hot encoding approach is applied to the survey categories to enable better feature rep- resentation for machine learning models. The results show that the Random Forest Classi- fier and Support Vector Machine achieved the highest accuracies in classifying survey papers based on their taxonomy. This research not only highlights trends in the publication of sur- veys but also offers an automated approach for classifying them, potentially aiding future re- search in organizing and categorizing survey literature efficiently. Keywords: Data Exploration ,Data Visualization , Classifiers ,LLM Survey ,TF-IDF 1 Introduction The advent of large-scale digital datasets has dra- matically reshaped the landscape of data-driven research, particularly in fields such as computer science, social science, and healthcare. Data- driven methods have become integral to uncover- ing meaningful insights, patterns, and trends fromvast amounts of unstructured information. This paper presents an analysis of survey data using various machine-learning techniques and provides a framework for understanding trends in research surveys. Specifically, this analysis focuses on ex- tracting valuable insights from survey datasets, in- cluding trend analysis, taxonomy distribution, and classification of papers using machine learning al- gorithms. The dataset used in this study includes survey papers characterized by fields such as the release date, paper title, taxonomy categories, and textual summaries. This analysis’s main goals are to identify temporal trends, categorize papers ac- cording to their taxonomy, and evaluate how well different machine learning models perform in the classification of survey papers. We implement a multi-step data analysis pipeline that includes data exploration, visualization, and machine learning- based classification. The first phase of this analysis focuses on explor- ing the dataset to identify trends over time. Using Python’s pandas library, we preprocess the dataset by extracting temporal features such as the year and month from the release dates of the survey papers. We then group the dataset by year and month to observe the frequency of surveys released over time. The ggplot-style visualizations help to identify any cyclic patterns, growth, or declines in survey releases. These visualizations serve as the foundation for identifying how interest in par- ticular research areas evolve over time, providing researchers with a clear understanding of temporal trends. In the second phase, we analyze the distribution of taxonomy categories proposed in the dataset. The taxonomy categories reflect the thematic areas of the survey papers. We use bar charts to visualize the distribution of these categories, allowing us to identify which taxonomies are dominant and which are underrepresented. For example, the "Trustwor- thy" category comprises 26 papers, representing a significant portion of the dataset. Next, we em- ploy machine learning techniques to classify papers based on their titles, summaries, and taxonomy cat- egories. We vectorize the textual aspects of the dataset, including the article titles and summaries, by using the TF-IDF (Term Frequency-Inverse Doc- ument Frequency) technique for the creation of a feature matrix. Additionally, we use one-hot en- coding for categorical fields, such as the taxonomy categories. This feature engineering step allows us to build a comprehensive feature matrix represent- ing the dataset in a numerical format that is suitable for machine learning models. Lastly, we use a several machine learning clas- sifiers, like Naive Bayes, Random Forests Classi- fier, Decision Tree, then Support Vector Machines (SVM), and Logistic Regression, to categorize the survey papers according to their taxonomy. Us- ing common classification metrics, including ac- curacy, recall, precision, and confusion matrices, we evaluate the performance of each model. Our results demonstrate varying degrees of effective- ness across these models, with Random Forests and SVM achieving higher classification accuracy compared to others. This analysis contributes to the field of survey paper categorization and data-driven research by offering a comprehensive methodology for extracting trends and insights from research datasets. The findings have the potential to guide future research efforts, highlighting emerging fields and presenting an accurate picture of the changing field of research interests. 2 Related Work A thorough overview of the integration of the Large Language Models (LLMs) with graph learning techniques is provided on LLMs for graphs X. Ren. et al. [ 1]. It categorizes existing methods into four unique designs, highlighting their strengths and limitations. The authors discuss the impressive per- formance of LLMs in natural language processing and their potential to enhance graph-centric tasks, such as link prediction and node classification. However, challenges like data sparsity, scalability, and limited interaction between agents and graph data persist. The survey aims to inspire future re- search and innovation in leveraging LLMs for more effective graph-related applications. X. Wang et al. [2] describes a thorough rundown of how Large Language Models (LLMs) and Knowledge Rep-resentation Learning (KRL) are integrated, yet it lacks depth in critical analysis. While it lists vari- ous models and datasets, it does not sufficiently ad- dress the limitations and challenges associated with these approaches, such as computational demands and data requirements. Additionally, the evaluation metrics discussed, while important, could benefit from a more nuanced discussion on their applica- bility across different contexts. The integration of Graph Representation Learn- ing (GRL) with Large Language Models (LLMs) is investigated in this survey Q Mao et al. [ 3], enhanc- ing contextual understanding. It categorizes compo- nents and techniques, discusses training strategies, and identifies future research directions, particu- larly in adapting LLMs to non-textual graph data and improving knowledge transfer across diverse graph structures. There remains a significant gap in systematically understanding how to effectively design and train models that leverage LLMs for non-textual graph data interpretation and knowl- edge transfer across diverse graph structures. R. Liu et al. [ 4] investigates the performance of Graph Neural Networks (GNNs) across various bench- mark datasets, emphasizing the importance of both graph structures and node features. It reveals that excluding node features significantly impacts GNN prediction performance, particularly in transductive tasks. The study highlights the necessity for com- prehensive benchmarking frameworks, as synthetic datasets may not accurately reflect real-world com- plexities. Additionally, it discusses the potential of new datasets, like DeezerEurope, and suggests that understanding dataset characteristics can guide advancements in self-supervised learning and data augmentation for GNNs. M. A. K. Raiaan et al. [5] explores the architecture, evolution, and trans- former models of LLMs like GPT and BERT, along with their practical applications in various sectors like healthcare, education, and business. The paper also discusses challenges related to LLM deploy- ment, such as security, ethical concerns, privacy, and future research directions. The paper high- lights key LLMs, such as GPT, BERT, PaLM, and LaMDA, explaining their architectures, training methods, and diverse applications. Y . Li et al. [ 6] presents a novel approach for learning effective representations from text- attributed graphs (TAGs) using a combination of the pre-trained language models (PLMs) and graph neural networks (GNNs). Existing methods for the self-supervised learning on TAGs often struggle to fully capture structural context or rely on labeled data. Grenade introduces two self-supervised learn- ing techniques, such as Graph-Centric Knowledge Alignment and Graph-Centric Contrastive Learn- ing, to address this. These techniques help cap- ture both the textual semantics and structural rela- tionships in the graph, improving performance in various downstream tasks such as node classifica- tion, link prediction and clustering. Extensive trials show that Grenade produces more generalizable representations than state-of-the-art techniques. A model centered on the integration of graph-based tasks with Large Language Models (LLMs) was proposed by J. Li et al. [ 7]. Although LLMs have shown success in natural language processing, their full potential when applied to graph data re- mains underexplored. To be more precise, difficul- ties include the fact that creating graph prompts is more difficult than creating language prompts, that matching downstream graph tasks to pre-training activities is more complicated, and that theoretical study is required to fully comprehend the function of graph prompts. Additionally, there is a lack of clear evaluation criteria for assessing the effective- ness of these prompts. T. Sen et al. [ 8] presents a novel approach to enhancing pre-trained language models (LMs) by integrating structured knowledge from knowledge graphs (KGs). The authors pro- pose two self-supervised learning tasks to enrich LMs: an entity-level masking scheme that uses KGs to select and mask key entities and a distractor- suppressed ranking task to improve learning by us- ing negative entity samples from KGs. This method incorporates structured knowledge implicitly dur- ing pre-training, making it more efficient and gener- alizable compared to existing approaches that rely on explicit KG integration during fine-tuning. Tests reveal that the model outperforms the benchmarks in a number of domains, such as knowledge base completion and question answering. D. C. Zhang et al. [ 9] presents an innovative method to integrate knowledge graphs with pre- trained language models, but it also has notable limitations. One criticism is that while the model claims efficiency and generalizability, its reliance on knowledge graphs during pre-training could limit applicability to domains lacking rich, well- structured knowledge graphs. Additionally, the proposed approach’s focus on entity-level mask-ing and distractor-based ranking might oversim- plify complex relational information, potentially overlooking subtler semantic nuances. Moreover, the performance improvements, though demon- strated on a few benchmarks, may not general- ize well across a broader range of tasks, par- ticularly in real-world settings where knowledge graph quality varies widely. Finally, the method’s computational requirements for pre-training may hinder its scalability for larger and more di- verse corpora. W. Ju et al. [ 10] introduces a method to integrate knowledge graphs with the pre-trained language models (PLMs) through two novel self-supervised tasks: entity-level masking and distractor-suppressed ranking. This aims to im- prove models’ ability to handle structured knowl- edge without needing explicit KG usage during fine-tuning, showing improved results in question answering and knowledge base tasks. However, the paper’s focus on pre-training with knowledge graphs might limit its generalizability to domains where structured knowledge is scarce or less or- ganized. Additionally, while the proposed tasks enhance entity representation, they might not fully capture more complex relationships or semantic nuances present in natural language, especially in less structured or informal text settings. Future work could explore incorporating more dynamic and heterogeneous data sources beyond knowledge graphs to address these limitations. 3 Methodology In this research work dataset has been loaded using the panda’s library. The dataset contains several information’s on various types of surveys such as release dates, titles, summaries, and assigned cate- gories. The following figure. 1 shows the method- ology of this research work. 3.1 Data Exploration The dataset is explored in two ways-first explore the dataset by analyzing the number of surveys grouped by their release date (Year-Month). Next, explore the dataset by analyzing the number of surveys grouped by their Paper ID (first part). 3.1.1 Explore the dataset by analyzing the num- ber of surveys grouped by their release date (Year-Month) Step-1 Conversion of Release Date to DateTime Format The Release Date column in the dataset is likely stored as text or a non-datetime format. For Figure 1: Methodology of the Taxonomy Classification accurate grouping and manipulation, this column has to convert to a datetime object. Step-2 Extracting Year and Month After the Re- lease Date column is in datetime format, it’s neces- sary to extract both the year and month from each release date for grouping purposes. Step-3 Grouping the Data by Year and Month: Then, the number of survey papers are grouped by both year and month for the analysis of trends over time. 3.1.2 Explore the dataset by analyzing the num- ber of surveys grouped by their Paper ID((first- part) Step-1 Splitting Paper ID The Paper ID column contains unique identifiers for each survey, format- ted with periods (e.g. 2023.0014). I extract the first part of this ID, which may represent a higher-level grouping of surveys. Step-2 Grouping by Paper ID (First Part) After ex- tracting the first part of the Paper ID, the dataset is grouped by this column to count how many surveys share the same Paper ID of its first part. 3.2 Data Visualization Visualizations provide critical insights into the sur- vey dataset, enabling the identification of trends, category distributions, and patterns across different dimensions. This Dataset is visualized into 3 ways with the help of “Matplotlib” Library. i.Visualize the data by survey trends over time (Year-Month)ii.Visualize the data by distribution of taxonomy categories iii.Visualize the data by survey trends by Paper ID (First Part) 3.3 Feature Extraction Feature extraction is a crucial step in the machine learning process, where raw text data is trans- formed into a format that can be processed by ma- chine learning algorithms. In this work feature extraction has been done in the following way. 3.3.1 Extract features from the dataset by trans- forming the Title and Summary text fields using TF-IDF vectorization Term Frequency-Inverse Document Frequency (TF- IDF) is a popular method used to convert text data into numerical representations by capturing the im- portance of words within a document relative to the entire corpus. TF-IDF Process: Term Frequency (TF) measures how frequently a word appears in a document. Words that appear more frequently are assigned higher values. tft,d=nt,d Number of terms in the document(1) Here, in the numerator, n is the number of times the term “t” appears in the document “d”. Thus, each document and term would have its own TF value. Inverse Document Frequency (IDF) reduces the weight of commonly occurring words (such as "the" or "is") across all documents, giving more importance to unique or rare words in the corpus. idft= logNumber of documents Number of documents with term t (2) 3.3.2 Encoding the Categories using one-hot en- coding One-hot encoding is a process of converting cate- gorical data into a binary matrix representation. In this dataset, the Categories column contains multi- ple comma-separated categories (e.g., "Cs", "AI"). The goal is to transform each category into its own binary feature, where the value is 1 if the survey belongs to that category and 0 otherwise. 3.3.3 Combining All Features After extracting features from the Title, Summary, and Categories, the next step is to combine them into a single feature matrix. This matrix will serve as the input for machine learning models. 3.4 Pre-Process Feature Matrix In this part, the feature matrix is prepared for ma- chine learning by applying normalization and label encoding. These preprocessing steps are crucial for ensuring that the features and labels are in a suitable format for machine learning models. 3.4.1 Normalization of the Feature Matrix Normalization is a common preprocessing step where features are scaled to a specific range (usu- ally between 0 and 1). This ensures that all features contribute equally to the model, and it prevents fea- tures with larger numerical values from dominating those with smaller values. For this task, the Min- MaxScaler is used which scales each feature to a range between 0 and 1. 3.4.2 Label Encoding of the Taxonomy Label encoding is used to convert the target labels (in this case, taxonomy categories) into a numerical format that machine learning models can process. Since the target labels are categorical, they must be transformed into numerical values. 3.5 Split Dataset The dataset is split into two parts—train and test. I keep 70% of the data for training and 30% for testing. 3.6 Employing Classifiers/Models I use five classifiers including logistic regression, decision tree, random forest, SVM, Naïve bayes toclassify the taxonomy of the papers. 3.7 Performance Evaluation I have used confusion matrix for measuring perfor- mance of our proposed model that is a tool which evaluates how well a classifier model performs. It provides the proportion of the model’s accurate and inaccurate predictions in comparison with the actual results. A standard performance indicator generated from the confusion matrix is accuracy, which is the proportion of correct predictions to all predictions. However, accuracy alone may not provide a complete picture of the model’s perfor- mance, as it can be biased by the distribution of the classes in the dataset. As a result, in addition to accuracy, other measures including F1 score, preci- sion and sensitivity can be employed. I must define the following to evaluate the system’s performance: 3.7.1 Accuracy: It refers to its ability to correctly distinguish between negative and positive cate- gories. This can be expressed as: Accuracy =TP+TN TP+TN+FP+FN×100% (3) 3.7.2 Sensitivity (Recall): The recall of a clas- sification system refers to its ability to correctly identify positive cases. Mathematically, recall can be calculated by taking the total correctly identified positive instances and it divides by the total posi- tive cases that actually exist. This can be expressed as: Sensitivity =TP TP+FN×100% (4) 3.7.3 Precision: Precision is a performance metric that computes the proportion of correctly classified instances that are positive out of every positive instance predicted by the categorization classifier. Mathematically, it is the fraction of true positive instances out of all cases classified as positive by the system or test, regardless of whether they are actually positive or negative. This can be expressed as: Precision =TP TP+FP×100% (5) 3.7.4 F1-score: A model’s accuracy can be evalu- ated using the F1 score, which takes both precision and recall into account. The F1 score is defined as the harmonic mean of recall and precision. A higher F1 score, ranging from 0 to 1, signifies better performance.This can be expressed as follows: F1 Score =2×recall×precision precision +recall(6) 4 Experimental Results and Evaluation 4.1 Results of Data Exploration and Visualization In this part it has been described about the data exploration and visualization 4.1.1 Survey Trends In Fig. 2, a line chart is created to visualize the number of surveys released each month. The x- axis represents the Year-Month while the y-axis represents the number of surveys. This exploration reveals seasonal patterns in survey releases. Peaks are observed at regular intervals, indicating that certain periods are more active for research and publication. Table 1: Survey Data Summary Year Month No. of Surv count 15.000000 15.000000 15.000000 mean 2022.800000 7.000000 9.600000 std 0.676123 3.505098 6.231258 min 2021.000000 1.000000 1.000000 25% 2023.000000 4.500000 4.000000 50% 2023.000000 7.000000 11.000000 75% 2023.000000 9.500000 14.500000 max 2024.000000 12.000000 21.000000 Table1 provides descriptive statistics for a dataset containing "Year," "Month," and "Number of Surveys" for 15 observations. The average year is 2022.8, with surveys mainly conducted around July (mean month 7), and an average of 9.6 surveys per entry. The highest number of surveys recorded was 21, while the lowest was 1, and most of the surveys were concentrated between 4 and 14.5 sur- veys (25th to 75th percentiles). The data spans from 2021 to 2024. 4.1.2 Taxonomy Distribution: The Fig. 3 shows the taxonomy distribution In Fig. 3, a bar chart is plotted to show the distribution of surveys across different taxonomy categories. The category "Trustworthy" has the highest number of papers, followed by other categories. This suggests a major research focus on building trustworthy cat- egory in the dataset.Table 2: Taxonomy Counts Taxonomy Count Trustworthy 26 Comprehensive 17 Prompting 17 Science 13 RecSys & IR 10 Multi-modal & Pre-training 9 Evaluation 9 Software Engineering 9 Adaptation Tuning 8 Robotics 8 Graphs 8 Others 5 Law 2 Finance 1 Education 1 Hardware Architecture 1 Table 2. lists topics with their respective frequen- cies under the "Taxonomy" category. "Trustwor- thy" is the most frequent topic with 26 entries, fol- lowed by "Comprehensive" and "Prompting" with 17 each. Less common topics include "Law" (2) and "Finance," "Education," and "Hardware Archi- tecture," each with 1 entry. 4.1.3 Paper ID Part Trends In Fig. 4, a line chart is used to visualize the trends based on the first part of the Paper ID. Certain Paper ID (First-Part) values has a signif- icantly higher number of surveys, indicating that some groups or institutions are more active in pub- lishing research. Table 3: Summary Statistics for Number of Surveys Statistic Value Count 19.000000 Mean 7.578947 Std 5.965789 Min 1.000000 25% 1.500000 50% 7.000000 75% 12.500000 Max 20.000000 Table 3 summarizes statistics for "Number of Surveys" based on Paper ID (First-Part) across 19 observations. The average number of surveys is Figure 2: Survey Trends Over Time (Year-Month) Figure 3: Distribution of taxonomy categories) 7.58, with a standard deviation of 5.97, indicating some variation. The range of surveys goes from a minimum of 1 to a maximum of 20, with the middle 50 4.2 Results of feature matrix The Fig. 5 shows the feature matrix of the dataset. 4.3 Results of Classifiers This research applies five distinct classifier mod- els—Logistic Regression, Decision Tree, Random Forest, SVM, and Naïve Bayes—to a test dataset for classifying the taxonomy of research papers. Each classifier is evaluated individually on the dataset, and performance results are collected sep- arately for comparison. Detailed analysis of the results is conducted, with confusion matrices con-structed for each model to assess accuracy, preci- sion, recall, and other performance metrics. The use of multiple classifiers provides a comprehen- sive understanding of how each model performs in classifying the dataset and give some insights into their strengths and weaknesses in handling the given task. The following Figures (6, 7, 8, 9, 10) shows the confusion matrix of the classifiers applied to the dataset of this research work. Table 4 presents the accuracy of different clas- sifiers. Random Forest has the highest accuracy at 50%, followed by Logistic Regression at 48%. Decision Tree and Naïve Bayes both have an ac- curacy of 41%, while SVM performs the lowest at 39%. The performance difference highlights that Random Forest and Logistic Regression are Figure 4: Distribution of taxonomy categories) Figure 5: Output of Feature Matrix) Figure 6: Confusion Matrix of Logistic Regression more effective for this task, possibly because Ran- dom Forest’s ensemble method reduces overfitting, while Logistic Regression may capture linear pat- terns well. SVM, despite being a powerful method, performs poorly here, likely due to the characteristics of the dataset. Decision Tree and Naïve Bayes perform Figure 7: Confusion Matrix of Support Vector Machine moderately. This suggests that Random Forest and Logistic Regression may be more effective for this specific dataset or classification task. Figure 8: Confusion Matrix of Random Forest Classifier Figure 9: Confusion Matrix of Decision Tree 5 Conclusion In this project, we explored the "LLM Survey" dataset using various data exploration, manipula- tion, and evaluation techniques. We applied five different classifiers like Logistic Regression, Sup- port Vector Machine (SVM), Random Forest, De- cision Tree, and Naïve Bayes to classify the tax- onomy of the papers. Each classifier was trained and evaluated using metrics like accuracy, preci- sion, recall, and confusion matrices. The models demonstrated varying levels of performance, with Random Forest and SVM showing relatively higher Table 4: Accuracy of Five classifiers Classifier Accuracy (%) Logistic Regression 48 SVM 39 Random Forest 50 Decision Tree 41 Naïve Bayes 41 Figure 10: Confusion Matrix of Gaussian Naive Bayes accuracy in comparison to others. By analyzing trends over time and understanding the distribu- tion of taxonomies, this work provided valuable insights into the dataset.