With the rapid advancement of computer and information technologies, a vast number of re- search papers are now available both online and offline. As new research fields continue to emerge, users face significant challenges in finding and categorizing papers of interest. To address these issues, this paper presents a re- search paper classification system that groups papers into meaningful categories based on shared topics. The system identifies represen- tative keywords from each paper and uses the Term Frequency-Inverse Document Frequency (TF-IDF) method to measure the importance of words and measure of how many times a word appears in a document. The logistic regression algorithm is then employed to classify papers with similar topics. 1 Introduction AI techniques have been widely applied to var- ious domains, such as images (He et al., 2016; Dosovitskiy, 2020), texts (Vaswani et al., 2017; De- vlin et al., 2018), and graphs (Kipf and Welling, 2016; Zhuang and Al Hasan, 2022). As a critical subset of AI techniques, Large Language Models (LLMs) have gained significant attention in recent years (Radford et al., 2018, 2019; Brown et al., 2020; Achiam et al., 2023; Bai et al., 2022; Team et al., 2023). Especially, more and more new be- ginners are interested in the research topics about LLMs. To learn the recent progress in this field, new beginners commonly will read survey papers about LLMs. Therefore, to facilitate their learn- ing, numerous survey papers on LLMs have been published in the last two years. However, a large amount of these survey papers can be overwhelm- ing, making it challenging for new beginners to read them efficiently. To embrace this challenge, in this project, we aim to explore and analyze the metadata of LLMs survey papers, providing in- sights to enhance their accessibility and understand- ing (Zhuang and Kennington, 2024). Specifically,we aim to classify the survey papers into different categories (topics) using logistic regression. Overall, our contributions can be summarized as follows: • Importing data • Data exploration • Preprocessing of data • Machine learning 2 Related Work A number of classifcation techniques have been used for document classification. (Barigou, 2018)divides Automatic document classifcation into two methods: supervised and unsupervised. (Pradeepa et al., 2024) proposes logistic regression for text classification and identifies two main advan- tages of logistic regression where it can naturally provide probabilities and extend to multi-class clas- sifcation problems Another advantage is that most of the methods used in logistic regression model analysis follow the same principles used in linear regression. (Shah et al., 2020) have compared logis- tic regression, random forest and K-nearest neigh- bour as classification algorithms for BBC news text classification and the authors decided to show the comparison based on five parameters namely pre- cision, accuracy, F1-score, support and confusion matrix.However, (Indra et al., 2016) specifically considers logistic regression to classify tweets into the selected topics. 3 Methodology 3.1 Data Exploration The research was conducted using a dataset of 144 different survey papers with 8 attributes as shown in Table 1. For these survey papers, a taxonomy was designed where each paper was assigned to a corre- sponding category within the taxonomy. To better understand the data, a line graph as illustrated in Figure 1 which shows that the survey papers in the dataset range from July 2021 to January 2024 and the number of survey papers has been increasing significantly. Furthermore, a bar graph for the taxonomy attribute was used to better understand the distribution of the classes as shown in Figure 2.With the data being rightly skewed and having more of the taxonomy of papers being Trustworthy(with 26 papers) The distribution indicates that the class is extremely im- balanced.Further more data distribution was anal- ysed using a bar graph in Figure 3 which shows the distribution of survey papers across different cate- gories with cs.CL being the most frequent category authors choose for their works. 2021-072022-082022-122023-022023-032023-042023-052023-062023-072023-082023-092023-102023-112023-122024-01 Date (Year-Month)0.02.55.07.510.012.515.017.520.0Number of Survey Released Trend of Survey Papers Over Months Figure 1: Line graph to showing trends of survey papers over months Trustworthy ComprehensivePromptingScience RecSys & IR Multi-modal & Pre-trainingEvaluation Software EngineeringAdaptation TuningRoboticsGraphs OthersLaw FinanceEducation Hardware Architecture T axonomy0510152025Number of PapersDistribution of the proposed taxonomy Figure 2: Distribution of the proposed taxonomy 3.2 Data Manipulation The data was manipulated to represent text data as a numerical matrix where feature matrices for both title and summary were created using the term frequency-inverse document frequency (TF-IDF). According to (Kim and Gil, 2019) describes TF- IDF as a technique used extensively to retrieve information and mine text to assess the significance of each word within a collection of documents. After establishing a feature matrix, we went cs.CL cs.AI cs.LG cs.SE cs.IRnancs.CY cs.CV cs.DL cs.SIcs.CR cs.HC cs.MA cs.NE cs.MM cs.DC cs.PF cs.RO cs.PL cs.AR Category020406080100CountFrequency of CategoriesFigure 3: Frequency of Categories further to preprocess the feature matrix which in- volved preparing the data to be in the right format for the machine learning models, the following ac- tivities were applied in the preprocessing of the feature matrix; normalizing the data, one hot en- coding, encoding. Further more, the dataset to be used for evaluation was split into two parts: train- ing and testing sets using a test size of 0.4.The splitting of data enabled the model to be trained on one part of the data (the training set) and evaluate its performance on unseen data (the testing set). 3.3 Data Evaluation Attribute Description Taxonomy Proposed taxonomy Title Paper title Summary Abstract of papers Authors Lists of author’s name Links Links of papers Paper ID Paper ID Categories Category Release Date First released date Table 1: Data attributes and description. A logistic regression classifier was employed and trained on the model and predicted on the test set and parameters namely accuracy, precision, F1-score was given with an accuracy of 0.22. Having noticed that the accuracy was very low, we went further to scale the data and handle the imbal- anced data since one class appeared to have more instances than another significantly. The technique of assigning class-weight to balanced was used and helped the model to pay more attention to the minority class.The accuracy was increased to 0.47. Furthermore, we went further to apply hyperpa- rameter tuning to optimise the parameters of the model and this led to an improved accuracy of 0.52. The results for the logistic regression with the hyper-tuned parameters are shown in Figure 4 with the performance metrics where accuracy of 0.52 is obtained. This figure shows that all the parameters for each of the individual classes are calculated. Other machine learning algorithms like SVM and Random Forest for comparison were also run but still gave lower accuracies and not considered. Adaptation TuningomprehensiveEvaluationGraphsLaw Multi-modal & Pre-trainingOthers Prompting ecSys & IRRobotics Science Software EngineeringTrustworthy Category0.00.20.40.60.81.0ScorePerformance Metrics by Category precision recall f1-score Figure 4: Graph showing the metrics of Logistic regres- sion 3.4 Conclusion, challenges and future work According to the logistic regression algorithm, most research papers appear to be from graphs research topics. However, a challenge of low ac- curacy was encountered even after handling class imbalance. For future works, this can be handled by employing more machine learning algorithms for classification for comparison with the logistic regression in order to have improved evaluation metrics. A APPENDIX