# Feature Learning via Correlation Analysis for Effective Duplicate Detection
## 1 Introduction  
The widespread adoption of software across various domains has led to a significant increase in the frequency and complexity of software bugs. Reports indicate that as software systems become more intricate, the number of submitted bug reports has grown exponentially, often overwhelming development teams [1]. For example, the Apache bug repository alone has accumulated over 40,000 bug reports, of which approximately 25% are identified as duplicates [2]. This duplication creates inefficiencies in bug tracking, leading to wasted resources and delayed issue resolution.  
Duplicate bug reports are particularly challenging for developers, as identifying duplicates is often a manual process prone to human error. Studies have shown that manualduplicate identification can result in error rates as high as 15% [3], further compounding the inefficiencies in project management and reducing developer productivity. Automating this process has the potential to streamline project workflows, reduce redundancy, and enhance overall efficiency.  
Prior research has explored various methods for automating duplicate bug detection. [PERSON] et al. [4] utilized word embeddings to vectorize bug reports and compute their similarities, achieving moderate success in duplicate detection. Similarly, [PERSON] et al. [5] proposed a binary classification model based on textual similarity, showing the viability of machine learning in this domain. Although these approaches have shown promise, they often face limitations such as dependency on specific datasets, lack of generalizability across projects, and suboptimal performance metrics.  
In this study, we present a novel approach to detecting duplicate bug reports by leveraging product- and component-level features extracted from bug repositories. The methodology involves preprocessing bug reports, selecting the most relevant features using a top-K feature selection algorithm [6], and training a machine learning model based on the Bidirectional Encoder Representations from Transformers (BERT) architecture [7]. The proposed model not only identifies duplicates with high accuracy but also provides robust generalizability across diverse open-source projects.  
To validate the effectiveness of the proposed approach, we conducted experiments on seven open-source projects: Apache, JDT, Platform, KDE, Core, Firefox, and Thunderbird. The model achieved detection accuracies of 91.41%, 88.66%, 86.08%, 92.94%, 90.68%, 88.25%, and 91.62%, respectively, showing significant improvements over baseline models such as CNN, LSTM, CNN-LSTM, Naive Bayes, and Random Forest. The performance gains, ranging from 32% to 41% compared to these baselines, were statistically significant based on rigorous validation [8; 9; 10].  
This study makes the following contributions:  
* Feature-Based Classification: By integrating product and component features with the BERT model and optimizing feature selection, the proposed approach achieves superior accuracy in duplicate detection across multiple datasets.
* Comprehensive Benchmarking: The model's performance is compared against established baselines across diverse projects, providing strong evidence for its robustness and scalability.  
By addressing the challenges of duplicate bug report detection with an innovative feature-learning approach, this study aims to enhance software maintenance practices, reduce redundant developer effort, and improve overall project efficiency.