## 1 Introduction  
Protein structure is a complex and hierarchical concept, encompassing several organizational levels that are critical for biological function. The primary structure, defined by the linear amino acid sequence encoded by DNA, determines the protein's folding and, ultimately, its three-dimensional conformation [1]. At the secondary structure level, recurring spatial motifs such as \(\alpha\) helices, \(\beta\) sheets, and random coils are formed and stabilized by hydrogen bonds between backbone amide and carbonyl groups [2]. These local conformations significantly influence the overall tertiary structure and, consequently, the biological activity of proteins [3]. Understanding protein structure is essential in biochemistry, molecular biology, and drug discovery. The well-established \"sequence-structure-function\" paradigm highlights the importance of predicting a protein's 3D structure from its amino acid sequence [4, 5]. Structural predictions enable the identification of drug-binding sites, protein-protein interactions, and conformational dynamics, facilitating the development of novel therapeutics [6, 7, 8]. Protein Secondary Structure Prediction (PSSP) combines experimental and computational techniques to infer local structural motifs. While X-ray crystallography and cryo-electron microscopy provide high-resolution data, they are labor-intensive and resource-demanding [9]. In contrast, advances in computational methods, particularly in machine learning (ML) and deep learning (DL), have substantially improved the prediction of protein structures from primary sequences [10]. CNNs have been employed to detect spatial hierarchies in amino acid sequences, contributing to fine-grained secondary structure predictions [11; 12]. RNN-based models such as LSTMs and their bidirectional variants (Bi-LSTMs) are capable of modeling long-range dependencies and sequential context [13], which are crucial for accurate structure inference. A transformative development in computational biology is the application of BERT (Bidirectional Encoder Representations from Transformers) models to protein sequences. BERT's architecture enables bidirectional context modeling, allowing for the simultaneous consideration of preceding and succeeding amino acids [14]. This ability enhances interpretability and facilitates the extraction of high-level features relevant to structure and function. Among these models, ProtBERT has shown promising results in PSSP tasks. Trained on large-scale protein sequence datasets, it captures both structural and functional properties and can be fine-tuned for specific bioinformatics applications [15]. However, the use of BERT-derived embeddings introduces computational challenges due to their high dimensionality and the variable lengths of protein sequences. Notably, the maximum token limit of 512 in BERT models imposes constraints on long protein sequences, often requiring truncation or sliding windows, which may result in the loss of important contextual information [16]. Furthermore, encoding the complexity of sequences comprising 20 amino acids in a model originally designed for natural language understanding necessitates careful adaptation. In PSSP, two widely adopted classification schemes are Q3 and Q8. The Q3 model simplifies predictions into three categories: helix (H), strand (E), and coil (C). Q8 expands this into eight categories: \(\alpha\) helix (H), \(\beta\) strand (E), turns (T), and various coil types (C, G, I, B, and S). While Q3 offers simplicity and higher per-residue accuracy, Q8 provides more nuanced structural information critical for advanced applications [4]. The choice between Q3 and Q8 depends on the required resolution for the intended biological or medical application. The comprehensive goals and specific contributions of this study are presented below.