# Enhanced Viral Genome Classification Using Large Language Models
## 2 Related Works  
Genomic sequences serve as the blueprint for all living organisms, carrying critical information for various biological processes. Analyzing these sequences is crucial for various important tasks, including detecting genetic variations. These applications rely on understanding the intricate details encoded within genomic sequences to unlock new insights and potential treatments. In recent years, technological advancements have significantly enhanced our ability to analyze genomic data. Techniques ranging from traditional machine learning to advanced deep learning models have played a crucial role in this progress. Machine learning algorithms have been employed to identify patterns and correlations within genomic data, facilitating tasks such as variant identification and disease susceptibility prediction. Meanwhile, deep learning models have pushed the boundaries further by automatically extracting complex features from large genomic datasets, which is particularly useful for analyzing high-dimensional data. Numerous studies have explored and adapted traditional machine learning (ML) techniques for classifying DNA sequences. Techniques such as SVM, DT, RF, NB, and KNN have been extensively utilized [5]. [PERSON] et al. [6] demonstrated that SVM achieved the highest accuracy of 96.3% in DNA sequence classification compared to other machine learning models. Similarly, [PERSON] et al. [7] classified the COVID-19 virus using RF, achieving an accuracy of 99.7%. While these models have proven effective in scenarios with limited data, they rely heavily on manual feature engineering and struggle to scale with larger datasets and longer sequences. This limitation has led to the adoption of deep learning models, particularly CNN, which have been shown to be powerful tools for DNA sequence classification. [PERSON] et al. investigated DNA sequence classification utilizing a 1D CNN, which leverages their kernel-based architecture to identify local nucleotide patterns. This approach achieved a 99.06% accuracy on a promoter dataset [8]. Meanwhile, [PERSON] et al. introduced a novel method for DNA sequence representation using a pseudo-convolutional approach, achieving 99.18% accuracy in classifying the COVID-19 virus [9]. Despite these successes, a significant challenge remains in capturing long-range dependencies due to the localized receptive fields of these models, suggesting the potential for more effective hybrid approaches. To address the issue of capturing sequential dependencies in DNA sequences, researchers have often employed recurrent neural networks (RNNs), including Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks. [PERSON] et al. applied hybrid models that combine CNN and LSTM architectures to classify a five-class virus dataset, achieving an accuracy of 93.16% [10]. The emergence of large language models (LLMs) has significantly enhanced the classification of DNA sequences by drawing analogies between biological sequences and natural languages. Models from the BERT and GPT families have been specifically adapted for bioinformatics applications, leading to the development of specialized models like DNABERT, Protein BERT, and DNAGPT. In the DNABERT [12] model, DNA sequences are treated as an analogy to language, with sequences divided into tokens using the k-mer tokenization method. In this framework, tokens are considered akin to words in natural language. DNABERT effectively captures DNA patterns and has been applied to various tasks, such as classification and predicting transcription factor binding sites. etc.,  
DNABERT-2 [13; 14], the advanced iteration of DNABERT, is tailored for multi-species genomic analysis. It employs masked language modeling techniques, where specific k-mers are masked, and the model is trained to predict these masked k-mers. This approach enables a deeper understanding of genomic sequences. DNABERT-2 is equipped for additional downstream tasks, including motif detection and cross-genomic comparisons etc.